[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 19:05:14.017287 27445 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.205.126:42775
I20260812 19:05:14.020197 27445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 19:05:14.022194 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:14.052692 27461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:14.053066 27455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:14.054231 27445 server_base.cc:1034] running on GCE node
W20260812 19:05:14.053112 27458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:15.289882 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:15.290226 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:15.290416 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561515290396 us; error 0 us; skew 500 ppm
I20260812 19:05:15.291683 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:15.298313 27445 webserver.cc:460] Webserver started at http://127.26.205.126:36319/ using document root <none> and password file <none>
I20260812 19:05:15.299788 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:15.300009 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:15.300688 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:15.308023 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/master-0-root/instance:
uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a"
format_stamp: "Formatted at 2026-08-12 19:05:15 on dist-test-slave-vpvm"
I20260812 19:05:15.318723 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.010s sys 0.001s
I20260812 19:05:15.326798 27471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:15.328879 27445 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260812 19:05:15.329339 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/master-0-root
uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a"
format_stamp: "Formatted at 2026-08-12 19:05:15 on dist-test-slave-vpvm"
I20260812 19:05:15.329689 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:15.375816 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:15.377743 27445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 19:05:15.378304 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:15.457151 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.126:42775
I20260812 19:05:15.457202 27552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.126:42775 every 8 connection(s)
I20260812 19:05:15.468168 27553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:15.497205 27553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: Bootstrap starting.
I20260812 19:05:15.505685 27553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:15.508344 27553 log.cc:824] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: Log is configured to *not* fsync() on all Append() calls
I20260812 19:05:15.513693 27553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: No bootstrap required, opened a new log
I20260812 19:05:15.538838 27553 raft_consensus.cc:348] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER }
I20260812 19:05:15.539644 27553 raft_consensus.cc:374] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:15.540041 27553 raft_consensus.cc:729] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f186ba9decb24fd99bdc6b7bf44d0f5a, State: Initialized, Role: FOLLOWER
I20260812 19:05:15.541453 27553 consensus_queue.cc:260] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER }
I20260812 19:05:15.542101 27553 raft_consensus.cc:388] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:05:15.542421 27553 raft_consensus.cc:482] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:05:15.542783 27553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:15.546909 27553 raft_consensus.cc:504] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER }
I20260812 19:05:15.548013 27553 leader_election.cc:302] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f186ba9decb24fd99bdc6b7bf44d0f5a; no voters:
I20260812 19:05:15.550557 27553 leader_election.cc:288] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:05:15.550936 27558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:15.554347 27558 raft_consensus.cc:686] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 1 LEADER]: Becoming Leader. State: Replica: f186ba9decb24fd99bdc6b7bf44d0f5a, State: Running, Role: LEADER
I20260812 19:05:15.555438 27558 consensus_queue.cc:237] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER }
I20260812 19:05:15.556782 27553 sys_catalog.cc:564] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:05:15.576742 27559 sys_catalog.cc:455] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f186ba9decb24fd99bdc6b7bf44d0f5a. Latest consensus state: current_term: 1 leader_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER } }
I20260812 19:05:15.576505 27560 sys_catalog.cc:455] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f186ba9decb24fd99bdc6b7bf44d0f5a" member_type: VOTER } }
I20260812 19:05:15.577554 27560 sys_catalog.cc:458] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:15.577531 27559 sys_catalog.cc:458] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:15.584008 27571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:05:15.606882 27571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:05:15.610167 27445 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 19:05:15.630649 27571 catalog_manager.cc:1347] Generated new cluster ID: 228aa5c31d5e4a6eb205a77c3510a503
I20260812 19:05:15.630970 27571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:05:15.653343 27571 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:05:15.655686 27571 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:05:15.675582 27571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: Generated new TSK 0
I20260812 19:05:15.677305 27571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:05:15.742178 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:15.756003 27585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:15.759760 27587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:15.762384 27590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:15.764334 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:15.765231 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:15.765492 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:15.765746 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561515765724 us; error 0 us; skew 500 ppm
I20260812 19:05:15.766665 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:15.777611 27445 webserver.cc:460] Webserver started at http://127.26.205.65:45519/ using document root <none> and password file <none>
I20260812 19:05:15.778468 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:15.778734 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:15.779086 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:15.781136 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-0-root/instance:
uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
format_stamp: "Formatted at 2026-08-12 19:05:15 on dist-test-slave-vpvm"
I20260812 19:05:15.789265 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.010s sys 0.000s
I20260812 19:05:15.795755 27603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:15.796682 27445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:05:15.797075 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
format_stamp: "Formatted at 2026-08-12 19:05:15 on dist-test-slave-vpvm"
I20260812 19:05:15.797494 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:15.842111 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:15.844491 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:15.848397 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:15.855854 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:15.856235 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:15.856573 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:15.856792 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.037474 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.65:36573
I20260812 19:05:16.037521 27701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.65:36573 every 8 connection(s)
I20260812 19:05:16.049356 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:16.066697 27707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:16.083566 27708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:16.093569 27710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:16.095181 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:16.096482 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:16.096760 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:16.096987 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561516096969 us; error 0 us; skew 500 ppm
I20260812 19:05:16.097757 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:16.101646 27702 heartbeater.cc:347] Connected to a master server at 127.26.205.126:42775
I20260812 19:05:16.102180 27702 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:16.103667 27702 heartbeater.cc:511] Master 127.26.205.126:42775 requested a full tablet report, sending...
I20260812 19:05:16.109277 27494 ts_manager.cc:194] Registered new tserver with Master: 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573)
I20260812 19:05:16.111142 27445 webserver.cc:460] Webserver started at http://127.26.205.66:37771/ using document root <none> and password file <none>
I20260812 19:05:16.111786 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:16.112116 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:16.112411 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:16.113554 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-1-root/instance:
uuid: "a3965785fdae4032a25d815b6b9db48f"
format_stamp: "Formatted at 2026-08-12 19:05:16 on dist-test-slave-vpvm"
I20260812 19:05:16.114024 27494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50690
I20260812 19:05:16.119002 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260812 19:05:16.122812 27718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.123622 27445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:05:16.123876 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
uuid: "a3965785fdae4032a25d815b6b9db48f"
format_stamp: "Formatted at 2026-08-12 19:05:16 on dist-test-slave-vpvm"
I20260812 19:05:16.124136 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:16.148185 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:16.149757 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:16.154333 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:16.158725 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:16.158954 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.159207 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:16.159355 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.332823 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.66:45897
I20260812 19:05:16.333115 27824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.66:45897 every 8 connection(s)
I20260812 19:05:16.341331 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:16.353840 27833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:16.357872 27834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:16.365978 27837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:16.367519 27825 heartbeater.cc:347] Connected to a master server at 127.26.205.126:42775
I20260812 19:05:16.367987 27825 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:16.368901 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:16.368891 27825 heartbeater.cc:511] Master 127.26.205.126:42775 requested a full tablet report, sending...
I20260812 19:05:16.369817 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:16.370041 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:16.370234 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561516370217 us; error 0 us; skew 500 ppm
I20260812 19:05:16.370814 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:16.371630 27494 ts_manager.cc:194] Registered new tserver with Master: a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:16.374173 27445 webserver.cc:460] Webserver started at http://127.26.205.67:36961/ using document root <none> and password file <none>
I20260812 19:05:16.374300 27494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50692
I20260812 19:05:16.374866 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:16.375124 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:16.375459 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:16.376672 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-2-root/instance:
uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
format_stamp: "Formatted at 2026-08-12 19:05:16 on dist-test-slave-vpvm"
I20260812 19:05:16.382006 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 19:05:16.386521 27847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.387374 27445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 19:05:16.387648 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
format_stamp: "Formatted at 2026-08-12 19:05:16 on dist-test-slave-vpvm"
I20260812 19:05:16.387930 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786561513794812-27445-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:16.404001 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:16.405687 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:16.410171 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:16.414139 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:16.414341 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.414594 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:16.414734 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:16.594640 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.67:44919
I20260812 19:05:16.596112 27956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.67:44919 every 8 connection(s)
I20260812 19:05:16.618944 27957 heartbeater.cc:347] Connected to a master server at 127.26.205.126:42775
I20260812 19:05:16.619407 27957 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:16.620414 27957 heartbeater.cc:511] Master 127.26.205.126:42775 requested a full tablet report, sending...
I20260812 19:05:16.623116 27494 ts_manager.cc:194] Registered new tserver with Master: 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:16.623524 27445 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023194963s
I20260812 19:05:16.625355 27494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50706
I20260812 19:05:16.626688 27445 test_util.cc:268] Using random seed: 1150378722
I20260812 19:05:16.708042 27494 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50720:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:05:16.713832 27494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:05:16.890712 27751 tablet_service.cc:1467] Processing CreateTablet for tablet 041990ee56244143b0c0312865e1354d (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260812 19:05:16.893410 27748 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcc52d4aa0f476ebab344294ef3dd05 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260812 19:05:16.894672 27756 tablet_service.cc:1467] Processing CreateTablet for tablet a8a3725fb6c94b14b1652f52801c15f2 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260812 19:05:16.895964 27753 tablet_service.cc:1467] Processing CreateTablet for tablet e3ba6d20f581488686bb5e9080454bdb (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260812 19:05:16.894650 27755 tablet_service.cc:1467] Processing CreateTablet for tablet e08d7111f104446281bebbb3c032ec53 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260812 19:05:16.896036 27757 tablet_service.cc:1467] Processing CreateTablet for tablet bfba69a506344440b85bbd9ef6381441 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260812 19:05:16.893062 27758 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2944d2325f41b1926121d266c5d87a (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260812 19:05:16.896265 27761 tablet_service.cc:1467] Processing CreateTablet for tablet f14c1846fd344977a77deac8f1ccee5b (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260812 19:05:16.900343 27756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a3725fb6c94b14b1652f52801c15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.902384 27753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ba6d20f581488686bb5e9080454bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.907907 27751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041990ee56244143b0c0312865e1354d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.907104 27747 tablet_service.cc:1467] Processing CreateTablet for tablet 424dea439cfe4118b5c71ab4ee40fdb1 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260812 19:05:16.908792 27748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcc52d4aa0f476ebab344294ef3dd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.924320 27747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424dea439cfe4118b5c71ab4ee40fdb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.926257 27755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08d7111f104446281bebbb3c032ec53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.929809 27758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2944d2325f41b1926121d266c5d87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.934902 27757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfba69a506344440b85bbd9ef6381441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.937177 27761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14c1846fd344977a77deac8f1ccee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.951112 27645 tablet_service.cc:1467] Processing CreateTablet for tablet f14c1846fd344977a77deac8f1ccee5b (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260812 19:05:16.956869 27642 tablet_service.cc:1467] Processing CreateTablet for tablet bfba69a506344440b85bbd9ef6381441 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260812 19:05:16.951014 27643 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2944d2325f41b1926121d266c5d87a (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260812 19:05:16.962317 27638 tablet_service.cc:1467] Processing CreateTablet for tablet e08d7111f104446281bebbb3c032ec53 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260812 19:05:16.974399 27895 tablet_service.cc:1467] Processing CreateTablet for tablet 5b2944d2325f41b1926121d266c5d87a (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260812 19:05:16.978045 27645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14c1846fd344977a77deac8f1ccee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.985215 27641 tablet_service.cc:1467] Processing CreateTablet for tablet a8a3725fb6c94b14b1652f52801c15f2 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260812 19:05:16.990548 27895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2944d2325f41b1926121d266c5d87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.994679 27642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfba69a506344440b85bbd9ef6381441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.996233 27643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b2944d2325f41b1926121d266c5d87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.001349 27892 tablet_service.cc:1467] Processing CreateTablet for tablet e08d7111f104446281bebbb3c032ec53 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260812 19:05:16.995107 27896 tablet_service.cc:1467] Processing CreateTablet for tablet f14c1846fd344977a77deac8f1ccee5b (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260812 19:05:17.012715 27638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08d7111f104446281bebbb3c032ec53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.014851 27896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14c1846fd344977a77deac8f1ccee5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:16.997682 27637 tablet_service.cc:1467] Processing CreateTablet for tablet e3ba6d20f581488686bb5e9080454bdb (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260812 19:05:17.010502 27633 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcc52d4aa0f476ebab344294ef3dd05 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260812 19:05:17.020386 27893 tablet_service.cc:1467] Processing CreateTablet for tablet a8a3725fb6c94b14b1652f52801c15f2 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260812 19:05:17.014820 27631 tablet_service.cc:1467] Processing CreateTablet for tablet 424dea439cfe4118b5c71ab4ee40fdb1 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260812 19:05:17.018198 27894 tablet_service.cc:1467] Processing CreateTablet for tablet bfba69a506344440b85bbd9ef6381441 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260812 19:05:17.008038 27635 tablet_service.cc:1467] Processing CreateTablet for tablet 041990ee56244143b0c0312865e1354d (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260812 19:05:17.033747 27892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08d7111f104446281bebbb3c032ec53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.033214 27641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a3725fb6c94b14b1652f52801c15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.026497 27889 tablet_service.cc:1467] Processing CreateTablet for tablet e3ba6d20f581488686bb5e9080454bdb (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260812 19:05:17.038329 27887 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcc52d4aa0f476ebab344294ef3dd05 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260812 19:05:17.041152 27637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ba6d20f581488686bb5e9080454bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.040194 27888 tablet_service.cc:1467] Processing CreateTablet for tablet 041990ee56244143b0c0312865e1354d (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260812 19:05:17.045226 27894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfba69a506344440b85bbd9ef6381441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.045430 27631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424dea439cfe4118b5c71ab4ee40fdb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.055548 27884 tablet_service.cc:1467] Processing CreateTablet for tablet 424dea439cfe4118b5c71ab4ee40fdb1 (DEFAULT_TABLE table=test-workload [id=3908d388e6a246119dfdee57b5afeab4]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260812 19:05:17.053274 27893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8a3725fb6c94b14b1652f52801c15f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.058338 27887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcc52d4aa0f476ebab344294ef3dd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.051256 27633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcc52d4aa0f476ebab344294ef3dd05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.061551 27888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041990ee56244143b0c0312865e1354d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.066946 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 041990ee56244143b0c0312865e1354d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.067299 27997 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.069425 27884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 424dea439cfe4118b5c71ab4ee40fdb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.078723 27889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3ba6d20f581488686bb5e9080454bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:17.125167 27702 heartbeater.cc:503] Master 127.26.205.126:42775 was elected leader, sending a full tablet report...
I20260812 19:05:17.264269 27999 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.274963 27997 tablet_bootstrap.cc:654] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.297196 28000 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.299736 27999 tablet_bootstrap.cc:654] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.306658 27997 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.308216 27997 ts_tablet_manager.cc:1397] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.266s user 0.017s sys 0.006s
I20260812 19:05:17.312407 27997 raft_consensus.cc:348] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.315030 27997 raft_consensus.cc:374] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.315326 27997 raft_consensus.cc:729] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.315706 27999 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.316089 27999 ts_tablet_manager.cc:1397] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.053s user 0.006s sys 0.016s
I20260812 19:05:17.316219 27997 consensus_queue.cc:260] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.318655 28000 tablet_bootstrap.cc:654] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.320184 27999 raft_consensus.cc:348] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:17.322530 27999 raft_consensus.cc:374] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.322932 27999 raft_consensus.cc:729] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.323853 27999 consensus_queue.cc:260] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:17.329674 27997 ts_tablet_manager.cc:1428] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.021s user 0.015s sys 0.004s
I20260812 19:05:17.330329 27825 heartbeater.cc:503] Master 127.26.205.126:42775 was elected leader, sending a full tablet report...
I20260812 19:05:17.330983 27997 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.339843 27999 ts_tablet_manager.cc:1428] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.024s user 0.014s sys 0.009s
I20260812 19:05:17.342082 28000 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.342954 28000 ts_tablet_manager.cc:1397] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.051s user 0.012s sys 0.018s
I20260812 19:05:17.344724 27999 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.347077 28000 raft_consensus.cc:348] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.347945 28000 raft_consensus.cc:374] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.348320 28000 raft_consensus.cc:729] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.350386 27997 tablet_bootstrap.cc:654] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
W20260812 19:05:17.354040 27826 tablet.cc:2369] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:05:17.354264 28000 consensus_queue.cc:260] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.361711 27999 tablet_bootstrap.cc:654] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.370390 27957 heartbeater.cc:503] Master 127.26.205.126:42775 was elected leader, sending a full tablet report...
I20260812 19:05:17.371716 28000 ts_tablet_manager.cc:1428] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.028s user 0.001s sys 0.011s
I20260812 19:05:17.373061 28000 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.376731 27997 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.384932 27997 ts_tablet_manager.cc:1397] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.012s
I20260812 19:05:17.386758 28000 tablet_bootstrap.cc:654] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.389664 27997 raft_consensus.cc:348] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.390921 27997 raft_consensus.cc:374] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.391214 27997 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.391948 27997 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.396801 27997 ts_tablet_manager.cc:1428] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260812 19:05:17.398353 28000 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.398921 28000 ts_tablet_manager.cc:1397] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.010s
I20260812 19:05:17.399540 27999 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.400175 27999 ts_tablet_manager.cc:1397] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.056s user 0.014s sys 0.009s
I20260812 19:05:17.402112 27997 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.405189 27999 raft_consensus.cc:348] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.406040 27999 raft_consensus.cc:374] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.406378 27999 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.408674 28000 raft_consensus.cc:348] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:17.409456 28000 raft_consensus.cc:374] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.407281 27999 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.411114 27997 tablet_bootstrap.cc:654] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.409737 28000 raft_consensus.cc:729] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.413772 27999 ts_tablet_manager.cc:1428] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.013s user 0.006s sys 0.005s
I20260812 19:05:17.413489 28000 consensus_queue.cc:260] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:17.414851 27999 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.416875 28000 ts_tablet_manager.cc:1428] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.018s user 0.007s sys 0.000s
I20260812 19:05:17.417870 28000 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.424945 27999 tablet_bootstrap.cc:654] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.430006 28000 tablet_bootstrap.cc:654] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.432695 27997 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.433187 27997 ts_tablet_manager.cc:1397] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.014s
I20260812 19:05:17.436486 27997 raft_consensus.cc:348] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.437356 27997 raft_consensus.cc:374] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.437731 27997 raft_consensus.cc:729] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.439025 27997 consensus_queue.cc:260] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.443167 27997 ts_tablet_manager.cc:1428] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.010s user 0.000s sys 0.008s
I20260812 19:05:17.443346 27999 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.444182 27997 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.450153 27999 ts_tablet_manager.cc:1397] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.036s user 0.015s sys 0.005s
I20260812 19:05:17.451893 28000 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.452551 27997 tablet_bootstrap.cc:654] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.452587 28000 ts_tablet_manager.cc:1397] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.035s user 0.017s sys 0.008s
I20260812 19:05:17.453438 27999 raft_consensus.cc:348] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.454367 27999 raft_consensus.cc:374] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.454798 27999 raft_consensus.cc:729] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.455649 27999 consensus_queue.cc:260] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.458773 27999 ts_tablet_manager.cc:1428] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260812 19:05:17.458500 28000 raft_consensus.cc:348] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.459352 28000 raft_consensus.cc:374] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.459822 28000 raft_consensus.cc:729] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.459985 27999 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.460784 28000 consensus_queue.cc:260] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.466693 28000 ts_tablet_manager.cc:1428] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.014s user 0.001s sys 0.009s
I20260812 19:05:17.471432 27999 tablet_bootstrap.cc:654] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.472855 28000 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.477286 27997 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.477736 27997 ts_tablet_manager.cc:1397] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.034s user 0.011s sys 0.013s
I20260812 19:05:17.482501 28000 tablet_bootstrap.cc:654] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.482100 27997 raft_consensus.cc:348] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.482967 27997 raft_consensus.cc:374] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.483355 27997 raft_consensus.cc:729] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.484659 27997 consensus_queue.cc:260] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.487314 27997 ts_tablet_manager.cc:1428] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260812 19:05:17.487426 27999 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.488052 27999 ts_tablet_manager.cc:1397] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.028s user 0.016s sys 0.000s
I20260812 19:05:17.488927 27997 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.492075 27999 raft_consensus.cc:348] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.492933 27999 raft_consensus.cc:374] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.493273 27999 raft_consensus.cc:729] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.494047 27999 consensus_queue.cc:260] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.497256 27999 ts_tablet_manager.cc:1428] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260812 19:05:17.498438 28005 raft_consensus.cc:482] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.498895 28005 raft_consensus.cc:504] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.501847 27999 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.502103 28005 leader_election.cc:288] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.519972 28000 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.520797 28000 ts_tablet_manager.cc:1397] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.048s user 0.016s sys 0.009s
I20260812 19:05:17.524500 27997 tablet_bootstrap.cc:654] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.532681 28000 raft_consensus.cc:348] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.534420 28000 raft_consensus.cc:374] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.534781 28000 raft_consensus.cc:729] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.536223 28000 consensus_queue.cc:260] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.538398 28005 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.539306 27999 tablet_bootstrap.cc:654] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.538910 28005 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.542229 28005 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.546880 28000 ts_tablet_manager.cc:1428] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.026s user 0.006s sys 0.006s
I20260812 19:05:17.548138 28000 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.566020 27997 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.566744 27997 ts_tablet_manager.cc:1397] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.078s user 0.006s sys 0.015s
I20260812 19:05:17.566743 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:17.570479 28000 tablet_bootstrap.cc:654] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
W20260812 19:05:17.570374 27607 leader_election.cc:341] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.569254 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:17.571408 27999 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.573889 27999 ts_tablet_manager.cc:1397] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.072s user 0.010s sys 0.012s
I20260812 19:05:17.573266 27788 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 0.
I20260812 19:05:17.576360 27607 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:17.576421 27997 raft_consensus.cc:348] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.577670 28005 raft_consensus.cc:2781] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:17.578128 28005 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:17.578497 28005 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:17.579241 27997 raft_consensus.cc:374] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.579957 27997 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.581079 27997 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.588692 27999 raft_consensus.cc:348] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.590857 27999 raft_consensus.cc:374] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.591984 27999 raft_consensus.cc:729] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.591471 27997 ts_tablet_manager.cc:1428] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.023s user 0.006s sys 0.005s
I20260812 19:05:17.595248 27999 consensus_queue.cc:260] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.596765 27997 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.592655 28005 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
W20260812 19:05:17.600031 27575 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:05:17.599905 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:17.601300 27914 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 0.
I20260812 19:05:17.601016 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:17.604848 27608 leader_election.cc:302] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters: a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:17.612812 28035 raft_consensus.cc:2781] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:17.616782 28005 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.613288 28035 raft_consensus.cc:482] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:05:17.618510 27608 leader_election.cc:341] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.618804 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:17.619666 27788 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:17.626204 27997 tablet_bootstrap.cc:654] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.626348 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:17.622951 28035 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:05:17.629411 27608 leader_election.cc:341] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.631055 27999 ts_tablet_manager.cc:1428] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.054s user 0.008s sys 0.013s
I20260812 19:05:17.632227 28000 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.634199 27999 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:17.637420 28000 ts_tablet_manager.cc:1397] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.090s user 0.007s sys 0.022s
I20260812 19:05:17.650437 28035 raft_consensus.cc:504] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.650033 27788 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:17.653527 27997 tablet_bootstrap.cc:492] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:17.654366 27997 ts_tablet_manager.cc:1397] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.058s user 0.014s sys 0.012s
I20260812 19:05:17.654970 28003 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.654822 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:17.655514 28003 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.656976 27607 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters: 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:17.658614 28035 leader_election.cc:288] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.659871 28003 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.660630 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:17.662155 27913 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:17.663085 28000 raft_consensus.cc:348] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.664158 28000 raft_consensus.cc:374] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.664449 28000 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.661379 28035 raft_consensus.cc:2781] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:17.665253 28000 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.676371 28000 ts_tablet_manager.cc:1428] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.032s user 0.003s sys 0.004s
I20260812 19:05:17.674593 27997 raft_consensus.cc:348] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.679062 27997 raft_consensus.cc:374] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:17.679379 27997 raft_consensus.cc:729] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:17.680111 27997 consensus_queue.cc:260] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.681645 27999 tablet_bootstrap.cc:654] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.681288 28000 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:17.674249 28035 raft_consensus.cc:686] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:17.697052 28035 consensus_queue.cc:237] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.703727 27997 ts_tablet_manager.cc:1428] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.049s user 0.004s sys 0.004s
W20260812 19:05:17.705245 27607 leader_election.cc:341] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.706746 28000 tablet_bootstrap.cc:654] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.707746 27913 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:17.711607 27997 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:17.714869 27608 leader_election.cc:302] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters: a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:17.716588 28036 raft_consensus.cc:2781] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:17.717216 28036 raft_consensus.cc:686] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:17.718792 28036 consensus_queue.cc:237] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.740855 27494 catalog_manager.cc:5495] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:17.741434 28003 raft_consensus.cc:482] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.743765 28003 raft_consensus.cc:504] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.747156 28003 leader_election.cc:288] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.755407 27997 tablet_bootstrap.cc:654] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:17.765693 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:17.768800 28010 raft_consensus.cc:482] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.769400 28010 raft_consensus.cc:504] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.770058 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:17.784562 28010 leader_election.cc:288] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:17.779606 27914 raft_consensus.cc:2454] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:17.796939 28036 raft_consensus.cc:482] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:17.806614 28036 raft_consensus.cc:504] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:17.846591 27726 leader_election.cc:302] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:17.849489 28036 leader_election.cc:288] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.848428 28003 raft_consensus.cc:2781] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:17.851649 28003 raft_consensus.cc:482] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:17.852123 28003 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:17.856676 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:17.850126 27913 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:17.859265 28000 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:17.865604 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
W20260812 19:05:17.873430 27607 leader_election.cc:341] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.881040 27726 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:17.879467 27914 raft_consensus.cc:2454] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 0.
I20260812 19:05:17.879132 28000 ts_tablet_manager.cc:1397] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.198s user 0.016s sys 0.013s
I20260812 19:05:17.886507 28003 raft_consensus.cc:504] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:17.884581 27999 tablet_bootstrap.cc:492] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:17.892239 27999 ts_tablet_manager.cc:1397] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.258s user 0.026s sys 0.031s
I20260812 19:05:17.895915 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
W20260812 19:05:17.917517 27721 leader_election.cc:341] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:17.919445 28003 leader_election.cc:288] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:17.896250 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:17.909677 27608 leader_election.cc:302] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters: a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.015537 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:17.920372 28003 raft_consensus.cc:2781] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.016513 28036 raft_consensus.cc:482] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.016956 28003 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:17.960134 28010 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.017362 28003 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.017652 28010 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.017158 28036 raft_consensus.cc:504] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
W20260812 19:05:17.935843 27721 leader_election.cc:341] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.021551 28035 raft_consensus.cc:2781] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.022131 28036 leader_election.cc:288] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.022230 28035 raft_consensus.cc:482] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.022130 28010 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:18.022670 28035 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:05:18.020309 27721 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:36573, user_credentials={real_user=slave}} blocked reactor thread for 89942.4us
W20260812 19:05:18.023363 27721 leader_election.cc:341] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.024529 27997 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:18.024991 27997 ts_tablet_manager.cc:1397] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.314s user 0.015s sys 0.019s
W20260812 19:05:18.024958 27608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44919, user_credentials={real_user=slave}} blocked reactor thread for 128473us
I20260812 19:05:18.026345 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.027431 28036 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.027371 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:18.028049 27997 raft_consensus.cc:348] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.028127 28036 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.029600 27997 raft_consensus.cc:374] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.029966 27997 raft_consensus.cc:729] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.029654 28035 raft_consensus.cc:504] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:18.030620 27997 consensus_queue.cc:260] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.031214 28036 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.032343 28035 leader_election.cc:288] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.032070 28003 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
W20260812 19:05:17.944823 27726 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44919, user_credentials={real_user=slave}} blocked reactor thread for 64174.5us
I20260812 19:05:18.035303 27997 ts_tablet_manager.cc:1428] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.010s user 0.002s sys 0.004s
I20260812 19:05:18.032462 28000 raft_consensus.cc:348] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.045615 28000 raft_consensus.cc:374] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.037079 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
W20260812 19:05:18.037247 27608 leader_election.cc:341] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.044484 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
W20260812 19:05:18.044389 27849 leader_election.cc:341] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.047873 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.045945 28000 raft_consensus.cc:729] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.049526 27913 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.057863 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.058674 27789 raft_consensus.cc:2454] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 0.
I20260812 19:05:18.049384 28003 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.059934 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.056491 28062 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.056916 27999 raft_consensus.cc:348] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.063506 27999 raft_consensus.cc:374] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.063975 27999 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.063723 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
W20260812 19:05:18.066879 27849 leader_election.cc:341] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.059443 28000 consensus_queue.cc:260] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
W20260812 19:05:18.066246 27608 leader_election.cc:341] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.068672 27912 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.068996 28000 ts_tablet_manager.cc:1428] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.179s user 0.027s sys 0.023s
I20260812 19:05:18.066388 28062 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.069900 27999 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.071343 27997 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:18.072911 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:18.075076 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.075706 27660 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:18.076658 27913 raft_consensus.cc:2454] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 1.
I20260812 19:05:18.076845 27721 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:18.077730 28060 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.079043 28062 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.079634 28000 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:18.079558 28003 raft_consensus.cc:2781] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.080543 28003 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.082337 28003 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.082875 27999 ts_tablet_manager.cc:1428] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.190s user 0.007s sys 0.010s
I20260812 19:05:18.084062 27999 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:18.079242 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
W20260812 19:05:18.086386 27721 leader_election.cc:341] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.078331 28060 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.087327 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.090911 27607 leader_election.cc:302] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters: 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:18.091696 27788 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 0.
I20260812 19:05:18.092975 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
W20260812 19:05:18.093266 27607 leader_election.cc:341] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.093760 27914 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:18.093993 28060 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.095355 28035 raft_consensus.cc:2781] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.096642 28035 raft_consensus.cc:482] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.097100 28035 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.097632 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.098440 27659 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:18.099483 27721 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:18.101539 27912 raft_consensus.cc:2454] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:18.102484 28060 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.102844 28060 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.103251 28060 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.103085 27608 leader_election.cc:302] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters: a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.103708 28035 raft_consensus.cc:504] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.104418 28036 raft_consensus.cc:2781] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.104934 28036 raft_consensus.cc:686] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:18.094820 27913 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.105886 28036 consensus_queue.cc:237] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:18.096645 27726 leader_election.cc:302] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:18.108795 28062 raft_consensus.cc:2781] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.123715 28035 leader_election.cc:288] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.125171 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.125324 27607 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters: 5079e4b2a9a243a99ee8c02bbd7de798
W20260812 19:05:18.126704 27608 leader_election.cc:341] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.128822 28035 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.137966 28035 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.138610 28035 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.138844 27999 tablet_bootstrap.cc:654] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.142174 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.142334 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.149923 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:18.153357 27913 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 1.
I20260812 19:05:18.150790 28062 raft_consensus.cc:686] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:18.156463 28060 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.155426 28003 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.161670 27784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:18.162427 27784 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.156632 28062 consensus_queue.cc:237] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.164511 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:18.165068 27659 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.166157 27789 raft_consensus.cc:2379] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5079e4b2a9a243a99ee8c02bbd7de798 in current term 1: Already voted for candidate a3965785fdae4032a25d815b6b9db48f in this term.
I20260812 19:05:18.166275 28000 tablet_bootstrap.cc:654] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.163161 27997 tablet_bootstrap.cc:654] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.172232 27852 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.174939 28010 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.177278 28066 raft_consensus.cc:482] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.177832 28066 raft_consensus.cc:504] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.178197 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:18.179803 28003 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.184293 27658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
W20260812 19:05:18.184744 27726 leader_election.cc:341] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.188784 27726 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:18.189649 27999 tablet_bootstrap.cc:492] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:18.190258 27999 ts_tablet_manager.cc:1397] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.106s user 0.007s sys 0.022s
I20260812 19:05:18.185720 28060 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.191991 28003 raft_consensus.cc:2781] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.192512 28003 raft_consensus.cc:686] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:18.195149 28005 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.197479 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.183007 28066 leader_election.cc:288] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
W20260812 19:05:18.198794 27608 consensus_peers.cc:487] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 -> Peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Couldn't send request to peer 5079e4b2a9a243a99ee8c02bbd7de798. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:05:18.198429 28003 consensus_queue.cc:237] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.203727 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:18.204051 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.204504 28005 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.205060 27912 raft_consensus.cc:2379] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3965785fdae4032a25d815b6b9db48f in current term 1: Already voted for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in this term.
W20260812 19:05:18.207101 27726 leader_election.cc:341] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.209656 28010 raft_consensus.cc:2726] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.215170 27659 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 1.
I20260812 19:05:18.216504 27721 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:18.218070 28066 raft_consensus.cc:2781] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.219197 28066 raft_consensus.cc:686] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:18.220074 28066 consensus_queue.cc:237] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.205363 28035 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.213687 27784 raft_consensus.cc:2454] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:18.226399 27660 raft_consensus.cc:2379] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3965785fdae4032a25d815b6b9db48f in current term 1: Already voted for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in this term.
I20260812 19:05:18.228158 27721 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:18.228820 27914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.206456 27491 catalog_manager.cc:5495] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:18.229337 27726 leader_election.cc:302] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:18.229455 28060 raft_consensus.cc:2726] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.229473 27913 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 1.
W20260812 19:05:18.230260 27608 leader_election.cc:341] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.227492 28035 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.230608 27491 catalog_manager.cc:5495] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:18.234788 28060 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.249114 27997 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:18.249683 27999 raft_consensus.cc:348] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.249702 27997 ts_tablet_manager.cc:1397] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.185s user 0.014s sys 0.013s
I20260812 19:05:18.253798 27999 raft_consensus.cc:374] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.256779 27607 leader_election.cc:302] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters: 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:18.258962 27784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:18.259843 27784 raft_consensus.cc:2379] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d9fe90437284f87ad2f7b0ea25766b5 in current term 1: Already voted for candidate a3965785fdae4032a25d815b6b9db48f in this term.
I20260812 19:05:18.261056 27852 leader_election.cc:302] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.261937 28035 raft_consensus.cc:2781] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.262750 28035 raft_consensus.cc:686] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:18.266589 28035 consensus_queue.cc:237] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.258328 27999 raft_consensus.cc:729] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.274405 27607 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5; no voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.275568 28005 raft_consensus.cc:2726] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.276696 28060 raft_consensus.cc:2726] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.277585 28000 tablet_bootstrap.cc:492] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:18.278316 28000 ts_tablet_manager.cc:1397] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.199s user 0.010s sys 0.015s
I20260812 19:05:18.278143 27999 consensus_queue.cc:260] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.282037 28077 raft_consensus.cc:2726] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.293653 28000 raft_consensus.cc:348] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.295569 28036 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260812 19:05:18.311996 27997 raft_consensus.cc:348] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.312608 27997 raft_consensus.cc:374] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.312832 27997 raft_consensus.cc:729] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.313407 27997 consensus_queue.cc:260] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.316241 27997 ts_tablet_manager.cc:1428] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.062s user 0.006s sys 0.000s
I20260812 19:05:18.323235 27999 ts_tablet_manager.cc:1428] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.133s user 0.012s sys 0.001s
I20260812 19:05:18.306452 28000 raft_consensus.cc:374] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.324421 27997 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: Bootstrap starting.
I20260812 19:05:18.324458 28000 raft_consensus.cc:729] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.329057 27999 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:18.328981 28000 consensus_queue.cc:260] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.332127 28000 ts_tablet_manager.cc:1428] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.053s user 0.006s sys 0.008s
I20260812 19:05:18.333295 28000 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:18.339792 27492 catalog_manager.cc:5495] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 0 to 1, leader changed from <none> to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 1 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:18.345880 28036 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.346530 28036 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.347909 28000 tablet_bootstrap.cc:654] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.349255 28036 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.352155 27492 catalog_manager.cc:5495] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 0 to 1, leader changed from <none> to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 1 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:18.353179 27784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.354734 28060 raft_consensus.cc:482] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.355393 28060 raft_consensus.cc:504] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.368369 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:18.369200 28060 leader_election.cc:288] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.369370 27912 raft_consensus.cc:2379] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d9fe90437284f87ad2f7b0ea25766b5 in current term 1: Already voted for candidate a3965785fdae4032a25d815b6b9db48f in this term.
I20260812 19:05:18.372013 28077 raft_consensus.cc:482] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.374249 27608 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5; no voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.374640 28077 raft_consensus.cc:504] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.377315 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
W20260812 19:05:18.379312 27721 leader_election.cc:341] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.380424 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.380898 28005 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.381093 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:18.381804 27912 raft_consensus.cc:2454] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 0.
I20260812 19:05:18.383091 27726 leader_election.cc:302] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:18.384716 28060 raft_consensus.cc:2781] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:18.385185 28060 raft_consensus.cc:482] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:18.385679 28060 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.385286 27494 catalog_manager.cc:5495] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
W20260812 19:05:18.387734 27849 leader_election.cc:341] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.388669 27997 tablet_bootstrap.cc:654] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.380378 27784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.397990 28077 leader_election.cc:288] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:18.403551 28077 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.407634 28077 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.410077 28077 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:18.411373 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.412241 27660 raft_consensus.cc:2379] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5079e4b2a9a243a99ee8c02bbd7de798 in current term 1: Already voted for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in this term.
I20260812 19:05:18.414320 28005 raft_consensus.cc:2726] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.414252 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.417791 27788 raft_consensus.cc:2379] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5079e4b2a9a243a99ee8c02bbd7de798 in current term 1: Already voted for candidate a3965785fdae4032a25d815b6b9db48f in this term.
I20260812 19:05:18.421396 27852 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.422876 28077 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.427438 28060 raft_consensus.cc:504] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.429116 27784 raft_consensus.cc:2379] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5079e4b2a9a243a99ee8c02bbd7de798 in current term 1: Already voted for candidate a3965785fdae4032a25d815b6b9db48f in this term.
I20260812 19:05:18.430730 28060 leader_election.cc:288] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:18.431339 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:18.432351 27997 tablet_bootstrap.cc:492] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: No bootstrap required, opened a new log
I20260812 19:05:18.432763 27997 ts_tablet_manager.cc:1397] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: Time spent bootstrapping tablet: real 0.108s user 0.013s sys 0.010s
I20260812 19:05:18.433039 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:18.431959 27852 leader_election.cc:302] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.434130 27913 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:05:18.443630 27721 leader_election.cc:341] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:18.441797 27913 raft_consensus.cc:2454] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 1.
I20260812 19:05:18.446841 28010 raft_consensus.cc:2726] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.451025 28000 tablet_bootstrap.cc:492] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:18.450835 27726 leader_election.cc:302] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters: 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:18.451694 28000 ts_tablet_manager.cc:1397] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.119s user 0.026s sys 0.020s
I20260812 19:05:18.455423 28000 raft_consensus.cc:348] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.456130 28000 raft_consensus.cc:374] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.456401 28000 raft_consensus.cc:729] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.456934 28060 raft_consensus.cc:2781] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:18.458921 28060 raft_consensus.cc:686] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:18.459687 28060 consensus_queue.cc:237] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.459896 28000 consensus_queue.cc:260] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.455819 27492 catalog_manager.cc:5495] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 0 to 1, leader changed from <none> to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 1 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:18.463797 28000 ts_tablet_manager.cc:1428] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260812 19:05:18.463708 27999 tablet_bootstrap.cc:654] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.464677 28000 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: Bootstrap starting.
I20260812 19:05:18.467770 28077 raft_consensus.cc:2726] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.492302 27997 raft_consensus.cc:348] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:18.493221 27997 raft_consensus.cc:374] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.493564 27997 raft_consensus.cc:729] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.494740 28000 tablet_bootstrap.cc:654] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.494318 27997 consensus_queue.cc:260] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:18.496690 27789 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.497848 27997 ts_tablet_manager.cc:1428] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: Time spent starting tablet: real 0.065s user 0.010s sys 0.005s
I20260812 19:05:18.511684 28035 consensus_queue.cc:1035] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.525476 28035 consensus_queue.cc:1035] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260812 19:05:18.537909 28077 raft_consensus.cc:482] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.538445 28077 raft_consensus.cc:504] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.539947 27494 catalog_manager.cc:5495] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 0 to 1, leader changed from <none> to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 1 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:18.540762 28077 leader_election.cc:288] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:18.541831 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.542629 27784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.543797 27784 raft_consensus.cc:2379] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5079e4b2a9a243a99ee8c02bbd7de798 in current term 1: Already voted for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in this term.
I20260812 19:05:18.545686 27852 leader_election.cc:302] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.547526 28010 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.566179 28010 raft_consensus.cc:2726] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.575824 28000 tablet_bootstrap.cc:492] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: No bootstrap required, opened a new log
I20260812 19:05:18.576294 28000 ts_tablet_manager.cc:1397] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent bootstrapping tablet: real 0.112s user 0.023s sys 0.009s
I20260812 19:05:18.577898 27999 tablet_bootstrap.cc:492] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:18.578497 27999 ts_tablet_manager.cc:1397] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.250s user 0.018s sys 0.017s
I20260812 19:05:18.579794 28000 raft_consensus.cc:348] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.580498 28000 raft_consensus.cc:374] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.580749 28000 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.581389 28000 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.581614 27999 raft_consensus.cc:348] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.582217 27999 raft_consensus.cc:374] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.582473 27999 raft_consensus.cc:729] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.583032 27999 consensus_queue.cc:260] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.583639 28000 ts_tablet_manager.cc:1428] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:05:18.585395 27999 ts_tablet_manager.cc:1428] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260812 19:05:18.586225 27999 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: Bootstrap starting.
I20260812 19:05:18.593501 27999 tablet_bootstrap.cc:654] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:18.604863 27999 tablet_bootstrap.cc:492] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: No bootstrap required, opened a new log
I20260812 19:05:18.606405 27999 ts_tablet_manager.cc:1397] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.011s
W20260812 19:05:18.609668 27575 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:05:18.611220 27999 raft_consensus.cc:348] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.610957 28060 consensus_queue.cc:1035] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.611717 27999 raft_consensus.cc:374] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:18.611980 27999 raft_consensus.cc:729] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Initialized, Role: FOLLOWER
I20260812 19:05:18.612563 27999 consensus_queue.cc:260] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.614615 27999 ts_tablet_manager.cc:1428] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:05:18.631965 27660 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.635452 28010 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:18.635955 28010 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:18.639240 28010 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:18.639474 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:18.639722 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" is_pre_election: true
I20260812 19:05:18.641662 27852 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798; no voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:18.642831 28010 raft_consensus.cc:2726] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:18.645916 28060 consensus_queue.cc:1035] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:18.688401 28060 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:18.704133 28062 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.744472 28035 consensus_queue.cc:1035] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:18.751394 28035 consensus_queue.cc:1035] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:18.769939 28062 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.791980 28062 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.922999 28062 consensus_queue.cc:1035] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:18.933373 27660 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:18.943665 28062 consensus_queue.cc:1035] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:18.965984 27912 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:05:19.614241 27575 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:05:20.103341 28128 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:20.103778 28128 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.105343 28128 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:20.106176 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:20.106457 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:20.106698 27789 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:20.107038 27912 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 1.
I20260812 19:05:20.107633 27607 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:20.108295 28128 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:05:20.108598 28128 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:20.108805 28128 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.113013 28128 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.114532 28128 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:20.115238 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:20.115540 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:20.115706 27789 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.116030 27912 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.119956 27789 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 2.
I20260812 19:05:20.120123 27912 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 2.
I20260812 19:05:20.120859 27607 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:20.121508 28128 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:20.122372 28128 raft_consensus.cc:686] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:20.123239 28128 consensus_queue.cc:237] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.130455 27492 catalog_manager.cc:5495] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 0 to 2, leader changed from <none> to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:20.274128 27789 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:54578
I20260812 19:05:20.274765 27789 raft_consensus.cc:595] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Received request to transfer leadership to TS 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:20.292596 27660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55410
I20260812 19:05:20.293079 27660 raft_consensus.cc:595] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:20.297166 27660 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55410
I20260812 19:05:20.297607 27660 raft_consensus.cc:595] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:20.306667 27789 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:54578
I20260812 19:05:20.307127 27789 raft_consensus.cc:595] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Received request to transfer leadership to TS 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:20.521232 28066 raft_consensus.cc:979] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:20.521714 28133 raft_consensus.cc:1067] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Signalling peer 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:20.523317 27912 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:44840
I20260812 19:05:20.523860 27912 raft_consensus.cc:482] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:20.524147 27912 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.528503 27912 raft_consensus.cc:504] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.530208 27912 leader_election.cc:288] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:20.531275 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:20.530926 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "424dea439cfe4118b5c71ab4ee40fdb1" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:20.531785 27789 raft_consensus.cc:3032] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:20.532146 27789 raft_consensus.cc:729] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:20.532135 27660 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.532114 27912 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260812 19:05:20.533046 27789 consensus_queue.cc:260] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.534471 27789 raft_consensus.cc:3037] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.534646 28128 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:20.538622 27660 raft_consensus.cc:2454] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:20.539896 27789 raft_consensus.cc:2454] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:20.540479 27849 leader_election.cc:302] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:20.541432 28151 raft_consensus.cc:2781] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:20.542724 28151 raft_consensus.cc:686] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Leader. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:20.543516 28151 consensus_queue.cc:237] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.552405 27492 catalog_manager.cc:5495] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 reported cstate change: term changed from 1 to 2, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67). New cstate: current_term: 2 leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:20.553252 27789 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260812 19:05:20.554668 28128 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:20.561741 28128 raft_consensus.cc:979] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:20.562146 28134 raft_consensus.cc:1067] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:20.563557 27789 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55614
I20260812 19:05:20.566306 27789 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:20.566677 27789 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:20.571205 27789 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.573244 27789 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:20.573475 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:20.573887 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:20.574107 27660 raft_consensus.cc:3032] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:20.574415 27660 raft_consensus.cc:729] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:20.574472 27912 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:20.575204 27660 consensus_queue.cc:260] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.576421 27660 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:20.580914 27912 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:20.582058 27726 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:20.582512 27660 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:20.582813 28133 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:20.583357 28133 raft_consensus.cc:686] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:20.584065 28133 consensus_queue.cc:237] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.591334 27491 catalog_manager.cc:5495] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 2 to 3, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 3 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:20.738343 28066 raft_consensus.cc:979] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:20.738740 28133 raft_consensus.cc:1067] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Signalling peer 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:20.739990 27660 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:50882
I20260812 19:05:20.740420 27660 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:20.740661 27660 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.744932 27660 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.746344 27660 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:20.747217 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:20.747308 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:20.747831 27789 raft_consensus.cc:3032] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:20.747936 27912 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.748111 27789 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:20.748769 27789 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.749675 27789 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.752761 27912 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 2.
I20260812 19:05:20.753773 27608 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:20.754540 28134 raft_consensus.cc:2781] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:20.754787 27789 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 2.
I20260812 19:05:20.755002 28134 raft_consensus.cc:686] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:20.755844 28134 consensus_queue.cc:237] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.762893 27491 catalog_manager.cc:5495] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 1 to 2, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:20.772579 28128 raft_consensus.cc:979] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:20.772989 28134 raft_consensus.cc:1067] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:20.774441 27789 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55614
I20260812 19:05:20.774878 27789 raft_consensus.cc:482] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:20.775153 27789 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.779394 27789 raft_consensus.cc:504] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.780810 27789 leader_election.cc:288] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:20.781684 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:20.781724 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:20.782467 27660 raft_consensus.cc:3032] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:20.782501 27912 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.782915 27660 raft_consensus.cc:729] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:20.783687 27660 consensus_queue.cc:260] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.784900 27660 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:20.789149 27912 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:20.790207 27726 leader_election.cc:302] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:20.790867 28133 raft_consensus.cc:2781] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:20.790812 27660 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:20.791406 28133 raft_consensus.cc:686] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:20.792148 28133 consensus_queue.cc:237] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:20.799374 27491 catalog_manager.cc:5495] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 1 to 2, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 2 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:21.014317 27660 raft_consensus.cc:1261] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.015480 28151 consensus_queue.cc:1035] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.023326 27789 raft_consensus.cc:1261] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.024828 28155 consensus_queue.cc:1035] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.029482 27912 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 19:05:21.031630 28066 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.038834 27660 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 19:05:21.040045 28066 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.324744 27912 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.326131 28134 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.330189 27789 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.331344 28134 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.398432 27912 raft_consensus.cc:1261] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.399770 28066 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:21.407661 27660 raft_consensus.cc:1261] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:21.409441 28066 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:05:22.816529 27694 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8080 after lost signal to thread 27450
W20260812 19:05:22.817245 27694 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8760 after lost signal to thread 27552
W20260812 19:05:22.817982 27694 debug-util.cc:398] Leaking SignalData structure 0x7b080015c660 after lost signal to thread 27701
W20260812 19:05:22.818681 27694 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef060 after lost signal to thread 27824
W20260812 19:05:22.819312 27694 debug-util.cc:398] Leaking SignalData structure 0x7b080003e860 after lost signal to thread 27956
I20260812 19:05:23.437148 27445 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20260812 19:05:23.438388 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:23.468220 27789 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:23.468909 27789 raft_consensus.cc:595] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Received request to transfer leadership to TS 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:23.473598 28133 raft_consensus.cc:979] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:23.474064 28184 raft_consensus.cc:1067] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Signalling peer 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:23.475780 27912 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:44840
I20260812 19:05:23.476310 27912 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:23.476621 27912 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.482595 27912 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.484081 27659 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:55422
I20260812 19:05:23.484648 27659 raft_consensus.cc:595] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Received request to transfer leadership to TS 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:23.485687 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:23.486006 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:23.486341 27660 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.486641 27789 raft_consensus.cc:3032] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:23.487022 27789 raft_consensus.cc:729] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:23.488216 27789 consensus_queue.cc:260] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.489423 27789 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.493073 27660 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:23.494310 27849 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:23.495049 28169 raft_consensus.cc:2781] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:23.495414 27912 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:23.496544 28169 raft_consensus.cc:686] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Leader. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:23.497303 28169 consensus_queue.cc:237] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.495128 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260812 19:05:23.501072 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:23.503198 27789 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:23.507738 27494 catalog_manager.cc:5495] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 reported cstate change: term changed from 1 to 2, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67). New cstate: current_term: 2 leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:23.861593 28200 raft_consensus.cc:979] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:23.862134 28176 raft_consensus.cc:1067] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Signalling peer 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:23.863822 27912 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:44836
I20260812 19:05:23.864251 27912 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:23.864492 27912 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.868643 27912 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.869904 27912 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:23.871083 27789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:23.871156 27660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:23.871733 27789 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.871971 27660 raft_consensus.cc:3032] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:23.872390 27660 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:23.873320 27660 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.874610 27660 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:23.879349 27789 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:23.881634 27660 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:23.886503 27852 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:23.890564 28169 raft_consensus.cc:2781] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:23.891052 28169 raft_consensus.cc:686] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Leader. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:23.891821 28169 consensus_queue.cc:237] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:23.904072 27494 catalog_manager.cc:5495] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 reported cstate change: term changed from 1 to 2, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67). New cstate: current_term: 2 leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:23.920694 27789 raft_consensus.cc:1261] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:23.922474 28169 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:23.952867 27660 raft_consensus.cc:1261] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:23.954979 28192 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:05:24.316538 27660 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:24.319658 28192 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260812 19:05:24.345651 27789 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:24.349004 28192 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:05:24.702055 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:24.709745 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:24.710341 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260812 19:05:25.542433 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800172060 after lost signal to thread 27450
W20260812 19:05:25.543144 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800172740 after lost signal to thread 27552
W20260812 19:05:25.543838 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800192860 after lost signal to thread 27701
W20260812 19:05:25.544492 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800090620 after lost signal to thread 27824
W20260812 19:05:25.545118 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08001519a0 after lost signal to thread 27956
I20260812 19:05:25.711176 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:25.716270 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:25.716691 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:26.717492 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:26.723097 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:26.723546 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:27.724316 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:27.729456 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:27.729840 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:29.318071 28232 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5079e4b2a9a243a99ee8c02bbd7de798)
I20260812 19:05:29.318635 28232 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:29.325328 28232 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:29.329337 28232 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3965785fdae4032a25d815b6b9db48f)
I20260812 19:05:29.329214 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:29.329932 28232 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:29.331662 28232 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:29.333251 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" is_pre_election: true
I20260812 19:05:29.365777 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:29.366091 27783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3965785fdae4032a25d815b6b9db48f" is_pre_election: true
I20260812 19:05:29.369915 27607 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5; no voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:29.371230 28232 raft_consensus.cc:2726] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:05:29.371165 27607 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5; no voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:29.371997 28232 raft_consensus.cc:2726] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:05:29.774883 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:29.779671 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:29.780074 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260812 19:05:30.256654 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08001d24c0 after lost signal to thread 27450
W20260812 19:05:30.257517 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800090600 after lost signal to thread 27552
W20260812 19:05:30.258450 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800172300 after lost signal to thread 27701
W20260812 19:05:30.259038 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0ee0 after lost signal to thread 27824
W20260812 19:05:30.259640 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5500 after lost signal to thread 27956
I20260812 19:05:30.780963 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:30.786348 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:30.786753 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:31.847527 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:31.852116 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:31.852633 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260812 19:05:32.520147 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5140 after lost signal to thread 27450
W20260812 19:05:32.520942 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800095d80 after lost signal to thread 27552
W20260812 19:05:32.521667 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08001c6520 after lost signal to thread 27701
W20260812 19:05:32.522365 27950 debug-util.cc:398] Leaking SignalData structure 0x7b0800152a80 after lost signal to thread 27824
W20260812 19:05:32.523162 27950 debug-util.cc:398] Leaking SignalData structure 0x7b08001722c0 after lost signal to thread 27956
I20260812 19:05:32.853425 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:32.858284 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:32.858649 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:33.859455 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:33.864141 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:33.864544 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:34.891685 27445 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20260812 19:05:34.896340 27659 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55410
I20260812 19:05:34.897333 27659 raft_consensus.cc:595] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:34.901687 27659 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55410
I20260812 19:05:34.902182 27659 raft_consensus.cc:595] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:34.906555 27659 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55410
I20260812 19:05:34.907030 27659 raft_consensus.cc:595] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:34.922917 27913 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:59590
I20260812 19:05:34.923492 27913 raft_consensus.cc:595] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:34.935544 27913 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
mode: GRACEFUL
new_leader_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:59590
I20260812 19:05:34.936050 27913 raft_consensus.cc:595] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Received request to transfer leadership to TS a3965785fdae4032a25d815b6b9db48f
I20260812 19:05:34.981524 28249 raft_consensus.cc:979] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:34.982062 28258 raft_consensus.cc:1067] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:34.983470 27788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55614
I20260812 19:05:34.983942 27788 raft_consensus.cc:482] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:34.984274 27788 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:34.990147 27788 raft_consensus.cc:504] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:34.992617 27788 leader_election.cc:288] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:34.993587 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:34.993566 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfba69a506344440b85bbd9ef6381441" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:34.994436 27913 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:34.994506 27659 raft_consensus.cc:3032] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:34.994848 27659 raft_consensus.cc:729] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:34.995507 27659 consensus_queue.cc:260] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:34.996513 27659 raft_consensus.cc:3037] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:34.999418 27913 raft_consensus.cc:2454] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:35.000571 27726 leader_election.cc:302] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:35.001361 28236 raft_consensus.cc:2781] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:35.001488 27659 raft_consensus.cc:2454] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:35.002403 28236 raft_consensus.cc:686] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:35.003300 28236 consensus_queue.cc:237] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.015196 27492 catalog_manager.cc:5495] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 1 to 2, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 2 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:35.023783 28251 raft_consensus.cc:979] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.024271 28251 raft_consensus.cc:1067] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.025866 27788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55624
I20260812 19:05:35.026403 27788 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:35.026705 27788 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.032195 27788 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.033576 27788 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:35.034428 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:35.034494 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:35.035033 27659 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.035126 27913 raft_consensus.cc:3032] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:35.035423 27913 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:35.036034 27913 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.036975 27913 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.039515 27659 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.040504 27721 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:35.041287 28236 raft_consensus.cc:2781] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:35.041640 28236 raft_consensus.cc:686] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:35.041749 27913 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.042454 28236 consensus_queue.cc:237] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.050333 27492 catalog_manager.cc:5495] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 2 to 3, leader changed from 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 3 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:35.216499 28264 raft_consensus.cc:979] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.216946 28251 raft_consensus.cc:1067] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.218236 27788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55624
I20260812 19:05:35.218744 27788 raft_consensus.cc:482] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:35.219023 27788 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.224041 27788 raft_consensus.cc:504] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.225456 27788 leader_election.cc:288] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:35.226315 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:35.226410 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcc52d4aa0f476ebab344294ef3dd05" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:35.226866 27659 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.226907 27913 raft_consensus.cc:3032] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:35.227227 27913 raft_consensus.cc:729] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:35.227844 27913 consensus_queue.cc:260] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.228762 27913 raft_consensus.cc:3037] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.231360 27659 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.232174 27721 leader_election.cc:302] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:35.232920 28259 raft_consensus.cc:2781] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:35.233304 28259 raft_consensus.cc:686] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:35.233372 27913 raft_consensus.cc:2454] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.234197 28259 consensus_queue.cc:237] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.241137 27492 catalog_manager.cc:5495] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 2 to 3, leader changed from 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 3 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:35.243808 28249 raft_consensus.cc:979] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.244215 28249 raft_consensus.cc:1067] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.245533 27788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55614
I20260812 19:05:35.246021 27788 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:35.246299 27788 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.251302 27788 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.252892 27788 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:35.253707 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:35.253870 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:35.254176 27913 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.254357 27659 raft_consensus.cc:3032] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:35.254631 27659 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:35.255205 27659 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.256083 27659 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:35.258262 27913 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.259122 27726 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:35.259824 28259 raft_consensus.cc:2781] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:35.260181 28259 raft_consensus.cc:686] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:35.260232 27659 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 3.
I20260812 19:05:35.260970 28259 consensus_queue.cc:237] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:35.266913 27492 catalog_manager.cc:5495] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 2 to 3, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 3 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:35.281153 28258 raft_consensus.cc:979] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: : Instructing follower a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.281556 28249 raft_consensus.cc:1067] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Signalling peer a3965785fdae4032a25d815b6b9db48f to start an election
I20260812 19:05:35.282836 27788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
from {username='slave'} at 127.0.0.1:55614
I20260812 19:05:35.283277 27788 raft_consensus.cc:482] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:35.283531 27788 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:35.288125 27788 raft_consensus.cc:504] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:35.289678 27788 leader_election.cc:288] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Requested vote from peers 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919), 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573)
I20260812 19:05:35.290483 27913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:35.290601 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14c1846fd344977a77deac8f1ccee5b" candidate_uuid: "a3965785fdae4032a25d815b6b9db48f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:35.290979 27913 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:35.291033 27659 raft_consensus.cc:3032] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:35.291334 27659 raft_consensus.cc:729] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:35.291896 27659 consensus_queue.cc:260] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:35.292734 27659 raft_consensus.cc:3037] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:35.295017 27913 raft_consensus.cc:2454] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:35.295830 27726 leader_election.cc:302] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5079e4b2a9a243a99ee8c02bbd7de798, a3965785fdae4032a25d815b6b9db48f; no voters:
I20260812 19:05:35.296402 28259 raft_consensus.cc:2781] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:35.296751 28259 raft_consensus.cc:686] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Becoming Leader. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:35.296836 27659 raft_consensus.cc:2454] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3965785fdae4032a25d815b6b9db48f in term 2.
I20260812 19:05:35.297640 28259 consensus_queue.cc:237] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } }
I20260812 19:05:35.304023 27492 catalog_manager.cc:5495] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f reported cstate change: term changed from 1 to 2, leader changed from 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65) to a3965785fdae4032a25d815b6b9db48f (127.26.205.66). New cstate: current_term: 2 leader_uuid: "a3965785fdae4032a25d815b6b9db48f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:35.429158 27659 raft_consensus.cc:1261] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:35.430250 28259 consensus_queue.cc:1035] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:35.441318 27913 raft_consensus.cc:1261] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:35.442483 28259 consensus_queue.cc:1035] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:35.490440 27659 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.491600 28259 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:35.499702 27913 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.500979 28259 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:35.654598 27659 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.655746 28259 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:35.663868 27912 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.665683 28283 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:35.704339 27912 raft_consensus.cc:1261] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:35.705772 28283 consensus_queue.cc:1035] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:35.716293 27659 raft_consensus.cc:1261] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:35.719540 28283 consensus_queue.cc:1035] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:35.756587 27659 raft_consensus.cc:1261] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.757820 28283 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:35.766527 27912 raft_consensus.cc:1261] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Refusing update from remote peer a3965785fdae4032a25d815b6b9db48f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:35.768275 28283 consensus_queue.cc:1035] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:37.966246 27445 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20260812 19:05:37.966863 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:37.973490 27788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "041990ee56244143b0c0312865e1354d"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:37.974049 27788 raft_consensus.cc:595] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Received request to transfer leadership to TS 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:37.976461 27788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:37.976982 27788 raft_consensus.cc:595] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Received request to transfer leadership to TS 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:37.979255 27788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:37.979923 27788 raft_consensus.cc:595] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Received request to transfer leadership to TS 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:37.982034 27788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:37.982483 27788 raft_consensus.cc:595] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Received request to transfer leadership to TS 5079e4b2a9a243a99ee8c02bbd7de798
I20260812 19:05:37.984495 27788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb"
dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
mode: GRACEFUL
new_leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:54590
I20260812 19:05:37.984935 27788 raft_consensus.cc:595] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Received request to transfer leadership to TS 4d9fe90437284f87ad2f7b0ea25766b5
I20260812 19:05:37.985860 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260812 19:05:37.986353 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:38.129462 28236 raft_consensus.cc:979] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.129914 28298 raft_consensus.cc:1067] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Signalling peer 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.131351 27659 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:50882
I20260812 19:05:38.131771 27659 raft_consensus.cc:482] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:38.132004 27659 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.136164 27659 raft_consensus.cc:504] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.138366 27659 leader_election.cc:288] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:38.138926 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:38.139178 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b2944d2325f41b1926121d266c5d87a" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:38.139461 27912 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.139778 27788 raft_consensus.cc:3032] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:05:38.140079 27788 raft_consensus.cc:729] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:38.140700 27788 consensus_queue.cc:260] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.141628 27788 raft_consensus.cc:3037] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.144021 27912 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.145026 27608 leader_election.cc:302] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:38.146509 28300 raft_consensus.cc:2781] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:05:38.146653 27788 raft_consensus.cc:2454] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.147451 28300 raft_consensus.cc:686] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:38.148326 28300 consensus_queue.cc:237] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.154201 27494 catalog_manager.cc:5495] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 3 to 4, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:38.238006 28236 raft_consensus.cc:979] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:38.238482 28298 raft_consensus.cc:1067] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Signalling peer 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:38.239908 27912 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "041990ee56244143b0c0312865e1354d"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:44840
I20260812 19:05:38.240317 27912 raft_consensus.cc:482] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:38.240592 27912 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:38.244637 27912 raft_consensus.cc:504] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.246443 27912 leader_election.cc:288] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:38.247339 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:38.247483 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041990ee56244143b0c0312865e1354d" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:38.247910 27659 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:38.247978 27788 raft_consensus.cc:3032] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:38.248286 27788 raft_consensus.cc:729] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:38.249053 27788 consensus_queue.cc:260] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.250367 27788 raft_consensus.cc:3037] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:38.254688 27659 raft_consensus.cc:2454] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:38.256125 27849 leader_election.cc:302] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:38.256909 27788 raft_consensus.cc:2454] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 2.
I20260812 19:05:38.257017 28299 raft_consensus.cc:2781] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:38.258526 28299 raft_consensus.cc:686] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Becoming Leader. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:38.259362 28299 consensus_queue.cc:237] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.266324 27494 catalog_manager.cc:5495] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 reported cstate change: term changed from 1 to 2, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67). New cstate: current_term: 2 leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:38.342554 28236 raft_consensus.cc:979] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.342969 28236 raft_consensus.cc:1067] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Signalling peer 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.344308 27659 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:50882
I20260812 19:05:38.344795 27659 raft_consensus.cc:482] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:38.345041 27659 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.349277 27659 raft_consensus.cc:504] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.350808 27659 leader_election.cc:288] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:38.351640 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:38.351754 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a3725fb6c94b14b1652f52801c15f2" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:38.352154 27788 raft_consensus.cc:3032] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:05:38.352241 27912 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.352411 27788 raft_consensus.cc:729] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:38.353080 27788 consensus_queue.cc:260] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.354239 27788 raft_consensus.cc:3037] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.356427 27912 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.357316 27608 leader_election.cc:302] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:38.357993 28300 raft_consensus.cc:2781] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:05:38.358469 28300 raft_consensus.cc:686] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:38.358997 27788 raft_consensus.cc:2454] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.359287 28300 consensus_queue.cc:237] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.366057 27494 catalog_manager.cc:5495] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 3 to 4, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:38.404009 28298 raft_consensus.cc:979] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:38.404461 28298 raft_consensus.cc:1067] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Signalling peer 5079e4b2a9a243a99ee8c02bbd7de798 to start an election
I20260812 19:05:38.405930 27912 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53"
dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
from {username='slave'} at 127.0.0.1:44840
I20260812 19:05:38.406406 27912 raft_consensus.cc:482] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:38.406667 27912 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:38.410725 27912 raft_consensus.cc:504] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.412164 27912 leader_election.cc:288] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 3 election: Requested vote from peers 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573), a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897)
I20260812 19:05:38.413026 27659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
I20260812 19:05:38.413211 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08d7111f104446281bebbb3c032ec53" candidate_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:38.413529 27659 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:38.413707 27788 raft_consensus.cc:3032] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:38.414047 27788 raft_consensus.cc:729] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:38.414595 27788 consensus_queue.cc:260] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.415383 27788 raft_consensus.cc:3037] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:38.418183 27659 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 3.
I20260812 19:05:38.419135 27849 leader_election.cc:302] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:38.419811 28299 raft_consensus.cc:2781] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:38.419963 27788 raft_consensus.cc:2454] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5079e4b2a9a243a99ee8c02bbd7de798 in term 3.
I20260812 19:05:38.420209 28299 raft_consensus.cc:686] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 LEADER]: Becoming Leader. State: Replica: 5079e4b2a9a243a99ee8c02bbd7de798, State: Running, Role: LEADER
I20260812 19:05:38.421170 28299 consensus_queue.cc:237] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.430152 27494 catalog_manager.cc:5495] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 reported cstate change: term changed from 2 to 3, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67). New cstate: current_term: 3 leader_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:38.436728 28236 raft_consensus.cc:979] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: : Instructing follower 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.437255 28298 raft_consensus.cc:1067] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Signalling peer 4d9fe90437284f87ad2f7b0ea25766b5 to start an election
I20260812 19:05:38.438697 27659 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb"
dest_uuid: "4d9fe90437284f87ad2f7b0ea25766b5"
from {username='slave'} at 127.0.0.1:50882
I20260812 19:05:38.439147 27659 raft_consensus.cc:482] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:38.439399 27659 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.444073 27659 raft_consensus.cc:504] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.445487 27659 leader_election.cc:288] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Requested vote from peers a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897), 5079e4b2a9a243a99ee8c02bbd7de798 (127.26.205.67:44919)
I20260812 19:05:38.446326 27788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a3965785fdae4032a25d815b6b9db48f"
I20260812 19:05:38.446488 27912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3ba6d20f581488686bb5e9080454bdb" candidate_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5079e4b2a9a243a99ee8c02bbd7de798"
I20260812 19:05:38.446893 27788 raft_consensus.cc:3032] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:05:38.446941 27912 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.447294 27788 raft_consensus.cc:729] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3965785fdae4032a25d815b6b9db48f, State: Running, Role: LEADER
I20260812 19:05:38.448063 27788 consensus_queue.cc:260] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.449219 27788 raft_consensus.cc:3037] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:05:38.453260 27912 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.454375 27608 leader_election.cc:302] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d9fe90437284f87ad2f7b0ea25766b5, 5079e4b2a9a243a99ee8c02bbd7de798; no voters:
I20260812 19:05:38.454906 27788 raft_consensus.cc:2454] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d9fe90437284f87ad2f7b0ea25766b5 in term 4.
I20260812 19:05:38.455107 28300 raft_consensus.cc:2781] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:05:38.455585 28300 raft_consensus.cc:686] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Becoming Leader. State: Replica: 4d9fe90437284f87ad2f7b0ea25766b5, State: Running, Role: LEADER
I20260812 19:05:38.456374 28300 consensus_queue.cc:237] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } }
I20260812 19:05:38.463114 27494 catalog_manager.cc:5495] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 reported cstate change: term changed from 3 to 4, leader changed from a3965785fdae4032a25d815b6b9db48f (127.26.205.66) to 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:38.599078 27912 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:05:38.600234 28300 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:05:38.609392 27788 raft_consensus.cc:1261] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:05:38.610935 28300 consensus_queue.cc:1035] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:05:38.712162 27788 raft_consensus.cc:1261] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:38.713310 28299 consensus_queue.cc:1035] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:38.724076 27659 raft_consensus.cc:1261] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:38.725589 28311 consensus_queue.cc:1035] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:05:38.809594 27788 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:05:38.811022 28300 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:05:38.820155 27912 raft_consensus.cc:1261] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:05:38.821785 28300 consensus_queue.cc:1035] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:05:38.868284 27912 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260812 19:05:38.869606 28300 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5079e4b2a9a243a99ee8c02bbd7de798" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:38.881524 27788 raft_consensus.cc:1261] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Refusing update from remote peer 4d9fe90437284f87ad2f7b0ea25766b5: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260812 19:05:38.883030 28301 consensus_queue.cc:1035] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:38.987186 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:38.991916 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:38.992287 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:38.999753 27788 raft_consensus.cc:1261] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:39.001034 28311 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3965785fdae4032a25d815b6b9db48f" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:39.006114 27659 raft_consensus.cc:1261] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Refusing update from remote peer 5079e4b2a9a243a99ee8c02bbd7de798: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:39.007491 28311 consensus_queue.cc:1035] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9fe90437284f87ad2f7b0ea25766b5" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 36573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:39.993113 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:39.998873 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:39.999392 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:41.000197 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:41.005236 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:41.005671 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:42.006455 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:42.011379 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:42.011783 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:43.012519 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:43.017175 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:43.017534 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:44.018326 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:44.023437 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:44.023834 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:45.024853 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:45.030448 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:45.030864 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:46.031635 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:46.036228 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:46.036654 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:47.037397 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:47.042258 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:47.042719 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:48.069713 27445 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20260812 19:05:48.070291 27445 tablet_server.cc:178] TabletServer@127.26.205.65:0 shutting down...
W20260812 19:05:48.091487 27721 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260812 19:05:48.096464 27721 consensus_peers.cc:487] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f -> Peer 4d9fe90437284f87ad2f7b0ea25766b5 (127.26.205.65:36573): Couldn't send request to peer 4d9fe90437284f87ad2f7b0ea25766b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:05:48.099802 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:05:48.100455 27445 tablet_replica.cc:330] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.101183 27445 raft_consensus.cc:2229] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.101788 27445 raft_consensus.cc:2258] T 041990ee56244143b0c0312865e1354d P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.104439 27445 tablet_replica.cc:330] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.104943 27445 raft_consensus.cc:2229] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.105369 27445 raft_consensus.cc:2258] T 424dea439cfe4118b5c71ab4ee40fdb1 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.107120 27445 tablet_replica.cc:330] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.107618 27445 raft_consensus.cc:2229] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.108484 27445 raft_consensus.cc:2258] T 5b2944d2325f41b1926121d266c5d87a P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.110342 27445 tablet_replica.cc:330] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.110919 27445 raft_consensus.cc:2229] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.111397 27445 raft_consensus.cc:2258] T e08d7111f104446281bebbb3c032ec53 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.113080 27445 tablet_replica.cc:330] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.113569 27445 raft_consensus.cc:2229] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.113989 27445 raft_consensus.cc:2258] T f14c1846fd344977a77deac8f1ccee5b P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.115753 27445 tablet_replica.cc:330] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.116215 27445 raft_consensus.cc:2229] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.117004 27445 raft_consensus.cc:2258] T a8a3725fb6c94b14b1652f52801c15f2 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.118690 27445 tablet_replica.cc:330] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.119118 27445 raft_consensus.cc:2229] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.119555 27445 raft_consensus.cc:2258] T bfba69a506344440b85bbd9ef6381441 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.121179 27445 tablet_replica.cc:330] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.121624 27445 raft_consensus.cc:2229] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.122009 27445 raft_consensus.cc:2258] T 0fcc52d4aa0f476ebab344294ef3dd05 P 4d9fe90437284f87ad2f7b0ea25766b5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.123478 27445 tablet_replica.cc:330] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5: stopping tablet replica
I20260812 19:05:48.123874 27445 raft_consensus.cc:2229] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.124711 27445 raft_consensus.cc:2258] T e3ba6d20f581488686bb5e9080454bdb P 4d9fe90437284f87ad2f7b0ea25766b5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.252435 27445 tablet_server.cc:194] TabletServer@127.26.205.65:0 shutdown complete.
I20260812 19:05:48.271379 27445 tablet_server.cc:178] TabletServer@127.26.205.66:0 shutting down...
I20260812 19:05:48.298485 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:05:48.299109 27445 tablet_replica.cc:330] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.299711 27445 raft_consensus.cc:2229] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.300166 27445 raft_consensus.cc:2258] T 424dea439cfe4118b5c71ab4ee40fdb1 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.301851 27445 tablet_replica.cc:330] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.302373 27445 raft_consensus.cc:2229] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.303175 27445 raft_consensus.cc:2258] T bfba69a506344440b85bbd9ef6381441 P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.304826 27445 tablet_replica.cc:330] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.305343 27445 raft_consensus.cc:2229] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.305747 27445 raft_consensus.cc:2258] T e08d7111f104446281bebbb3c032ec53 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.307399 27445 tablet_replica.cc:330] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.307840 27445 raft_consensus.cc:2229] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.308220 27445 raft_consensus.cc:2258] T 5b2944d2325f41b1926121d266c5d87a P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.309861 27445 tablet_replica.cc:330] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.310392 27445 raft_consensus.cc:2229] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 3 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.311156 27445 raft_consensus.cc:2258] T 0fcc52d4aa0f476ebab344294ef3dd05 P a3965785fdae4032a25d815b6b9db48f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.312749 27445 tablet_replica.cc:330] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.313191 27445 raft_consensus.cc:2229] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.313561 27445 raft_consensus.cc:2258] T e3ba6d20f581488686bb5e9080454bdb P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.315136 27445 tablet_replica.cc:330] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.315560 27445 raft_consensus.cc:2229] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.315944 27445 raft_consensus.cc:2258] T 041990ee56244143b0c0312865e1354d P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.317490 27445 tablet_replica.cc:330] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.317983 27445 raft_consensus.cc:2229] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.318775 27445 raft_consensus.cc:2258] T f14c1846fd344977a77deac8f1ccee5b P a3965785fdae4032a25d815b6b9db48f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.320322 27445 tablet_replica.cc:330] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f: stopping tablet replica
I20260812 19:05:48.320727 27445 raft_consensus.cc:2229] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.321074 27445 raft_consensus.cc:2258] T a8a3725fb6c94b14b1652f52801c15f2 P a3965785fdae4032a25d815b6b9db48f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.350553 27445 tablet_server.cc:194] TabletServer@127.26.205.66:0 shutdown complete.
W20260812 19:05:48.350464 27852 consensus_peers.cc:487] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 -> Peer a3965785fdae4032a25d815b6b9db48f (127.26.205.66:45897): Couldn't send request to peer a3965785fdae4032a25d815b6b9db48f. Status: Network error: Client connection negotiation failed: client connection to 127.26.205.66:45897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 19:05:48.367434 27445 tablet_server.cc:178] TabletServer@127.26.205.67:0 shutting down...
I20260812 19:05:48.391979 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:05:48.392608 27445 tablet_replica.cc:330] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.393263 27445 raft_consensus.cc:2229] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.393693 27445 raft_consensus.cc:2258] T bfba69a506344440b85bbd9ef6381441 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.395505 27445 tablet_replica.cc:330] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.396019 27445 raft_consensus.cc:2229] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.396806 27445 raft_consensus.cc:2258] T 424dea439cfe4118b5c71ab4ee40fdb1 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.398517 27445 tablet_replica.cc:330] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.399008 27445 raft_consensus.cc:2229] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.399750 27445 raft_consensus.cc:2258] T e08d7111f104446281bebbb3c032ec53 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.401365 27445 tablet_replica.cc:330] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.401820 27445 raft_consensus.cc:2229] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.402277 27445 raft_consensus.cc:2258] T 5b2944d2325f41b1926121d266c5d87a P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.403851 27445 tablet_replica.cc:330] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.404292 27445 raft_consensus.cc:2229] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.405052 27445 raft_consensus.cc:2258] T 041990ee56244143b0c0312865e1354d P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.406651 27445 tablet_replica.cc:330] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.407287 27445 raft_consensus.cc:2229] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.407686 27445 raft_consensus.cc:2258] T a8a3725fb6c94b14b1652f52801c15f2 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.409191 27445 tablet_replica.cc:330] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.409618 27445 raft_consensus.cc:2229] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.410029 27445 raft_consensus.cc:2258] T f14c1846fd344977a77deac8f1ccee5b P 5079e4b2a9a243a99ee8c02bbd7de798 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.411608 27445 tablet_replica.cc:330] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.412024 27445 raft_consensus.cc:2229] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.412392 27445 raft_consensus.cc:2258] T e3ba6d20f581488686bb5e9080454bdb P 5079e4b2a9a243a99ee8c02bbd7de798 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.413808 27445 tablet_replica.cc:330] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798: stopping tablet replica
I20260812 19:05:48.414247 27445 raft_consensus.cc:2229] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:05:48.414620 27445 raft_consensus.cc:2258] T 0fcc52d4aa0f476ebab344294ef3dd05 P 5079e4b2a9a243a99ee8c02bbd7de798 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.467937 27445 tablet_server.cc:194] TabletServer@127.26.205.67:0 shutdown complete.
I20260812 19:05:48.486975 27445 master.cc:537] Master@127.26.205.126:42775 shutting down...
I20260812 19:05:48.681327 27445 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:05:48.681911 27445 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:05:48.682349 27445 tablet_replica.cc:330] T 00000000000000000000000000000000 P f186ba9decb24fd99bdc6b7bf44d0f5a: stopping tablet replica
I20260812 19:05:48.701061 27445 master.cc:552] Master@127.26.205.126:42775 shutdown complete.
4d9fe90437284f87ad2f7b0ea25766b5 4
5079e4b2a9a243a99ee8c02bbd7de798 1
a3965785fdae4032a25d815b6b9db48f 4
4d9fe90437284f87ad2f7b0ea25766b5 3
5079e4b2a9a243a99ee8c02bbd7de798 3
a3965785fdae4032a25d815b6b9db48f 3
4d9fe90437284f87ad2f7b0ea25766b5 0
5079e4b2a9a243a99ee8c02bbd7de798 1
a3965785fdae4032a25d815b6b9db48f 8
4d9fe90437284f87ad2f7b0ea25766b5 3
5079e4b2a9a243a99ee8c02bbd7de798 3
a3965785fdae4032a25d815b6b9db48f 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (34730 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260812 19:05:48.742547 27445 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.205.126:34053
I20260812 19:05:48.743592 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:48.749207 28355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:48.749847 28356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:48.750663 28359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:48.751091 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:48.752202 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:48.752388 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:48.752506 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561548752490 us; error 0 us; skew 500 ppm
I20260812 19:05:48.752957 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:48.760886 27445 webserver.cc:460] Webserver started at http://127.26.205.126:42441/ using document root <none> and password file <none>
I20260812 19:05:48.761360 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:48.761521 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:48.761742 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:48.762930 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/master-0-root/instance:
uuid: "c7d60abd20a8465c8cea95d10595273f"
format_stamp: "Formatted at 2026-08-12 19:05:48 on dist-test-slave-vpvm"
I20260812 19:05:48.767323 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 19:05:48.770665 28364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:48.771407 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:05:48.771675 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/master-0-root
uuid: "c7d60abd20a8465c8cea95d10595273f"
format_stamp: "Formatted at 2026-08-12 19:05:48 on dist-test-slave-vpvm"
I20260812 19:05:48.771957 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:48.795176 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:48.796656 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:48.833379 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.126:34053
I20260812 19:05:48.833490 28446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.126:34053 every 8 connection(s)
I20260812 19:05:48.837195 28449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:48.846760 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f: Bootstrap starting.
I20260812 19:05:48.851184 28449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:48.855406 28449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f: No bootstrap required, opened a new log
I20260812 19:05:48.857343 28449 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER }
I20260812 19:05:48.857797 28449 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:48.858039 28449 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d60abd20a8465c8cea95d10595273f, State: Initialized, Role: FOLLOWER
I20260812 19:05:48.858577 28449 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER }
I20260812 19:05:48.858999 28449 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:05:48.859217 28449 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:05:48.859472 28449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:48.863255 28449 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER }
I20260812 19:05:48.863732 28449 leader_election.cc:302] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d60abd20a8465c8cea95d10595273f; no voters:
I20260812 19:05:48.864809 28449 leader_election.cc:288] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:05:48.865190 28453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:48.866541 28453 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 1 LEADER]: Becoming Leader. State: Replica: c7d60abd20a8465c8cea95d10595273f, State: Running, Role: LEADER
I20260812 19:05:48.867115 28453 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER }
I20260812 19:05:48.867691 28449 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:05:48.871376 28455 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d60abd20a8465c8cea95d10595273f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER } }
I20260812 19:05:48.872740 28455 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:48.872211 28458 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d60abd20a8465c8cea95d10595273f. Latest consensus state: current_term: 1 leader_uuid: "c7d60abd20a8465c8cea95d10595273f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d60abd20a8465c8cea95d10595273f" member_type: VOTER } }
I20260812 19:05:48.873349 28458 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [sys.catalog]: This master's current role is: LEADER
I20260812 19:05:48.880419 28469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:05:48.885300 28469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:05:48.886873 27445 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 19:05:48.893554 28469 catalog_manager.cc:1347] Generated new cluster ID: eb7caa6cd93143099912d4100215a6f1
I20260812 19:05:48.893826 28469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:05:48.920068 28469 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:05:48.921653 28469 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:05:48.932880 28469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f: Generated new TSK 0
I20260812 19:05:48.933547 28469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:05:48.953141 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:48.959174 28480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:48.959767 28481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:48.961390 28484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:48.962007 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:48.962972 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:48.963157 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:48.963272 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561548963264 us; error 0 us; skew 500 ppm
I20260812 19:05:48.963706 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:48.965938 27445 webserver.cc:460] Webserver started at http://127.26.205.65:46365/ using document root <none> and password file <none>
I20260812 19:05:48.966420 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:48.966573 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:48.966782 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:48.967922 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-0-root/instance:
uuid: "4e11a4f6072b426693a6664f95af0907"
format_stamp: "Formatted at 2026-08-12 19:05:48 on dist-test-slave-vpvm"
I20260812 19:05:48.972389 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 19:05:48.975557 28490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:48.976220 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 19:05:48.976454 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
uuid: "4e11a4f6072b426693a6664f95af0907"
format_stamp: "Formatted at 2026-08-12 19:05:48 on dist-test-slave-vpvm"
I20260812 19:05:48.976699 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:48.990512 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:48.991622 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:48.992969 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:48.995126 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:48.995302 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:48.995527 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:48.995687 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.034801 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.65:44075
I20260812 19:05:49.034933 28595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.65:44075 every 8 connection(s)
I20260812 19:05:49.039350 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:49.046494 28603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:49.048269 28604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:49.051214 28606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:49.051992 27445 server_base.cc:1034] running on GCE node
I20260812 19:05:49.053611 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:49.053855 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:49.054082 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561549054066 us; error 0 us; skew 500 ppm
I20260812 19:05:49.054706 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:49.056798 28596 heartbeater.cc:347] Connected to a master server at 127.26.205.126:34053
I20260812 19:05:49.057126 28596 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:49.057498 27445 webserver.cc:460] Webserver started at http://127.26.205.66:46375/ using document root <none> and password file <none>
I20260812 19:05:49.057879 28596 heartbeater.cc:511] Master 127.26.205.126:34053 requested a full tablet report, sending...
I20260812 19:05:49.058027 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:49.058277 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:49.058521 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:49.059813 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-1-root/instance:
uuid: "144c38de478a44859c4593c766e5b1da"
format_stamp: "Formatted at 2026-08-12 19:05:49 on dist-test-slave-vpvm"
I20260812 19:05:49.059969 28392 ts_manager.cc:194] Registered new tserver with Master: 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075)
I20260812 19:05:49.061872 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37170
I20260812 19:05:49.064900 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 19:05:49.068261 28612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.069080 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 19:05:49.069372 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
uuid: "144c38de478a44859c4593c766e5b1da"
format_stamp: "Formatted at 2026-08-12 19:05:49 on dist-test-slave-vpvm"
I20260812 19:05:49.069623 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:49.097826 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:49.099012 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:49.100517 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:49.102733 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:49.102952 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.103232 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:49.103377 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.142356 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.66:41381
I20260812 19:05:49.142442 28711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.66:41381 every 8 connection(s)
I20260812 19:05:49.146817 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:05:49.153443 28715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:05:49.154265 28718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:49.157567 27445 server_base.cc:1034] running on GCE node
W20260812 19:05:49.158223 28724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:05:49.159114 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:05:49.159312 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:05:49.159472 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561549159459 us; error 0 us; skew 500 ppm
I20260812 19:05:49.160157 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:05:49.161583 28712 heartbeater.cc:347] Connected to a master server at 127.26.205.126:34053
I20260812 19:05:49.161994 28712 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:49.162820 28712 heartbeater.cc:511] Master 127.26.205.126:34053 requested a full tablet report, sending...
I20260812 19:05:49.163036 27445 webserver.cc:460] Webserver started at http://127.26.205.67:38477/ using document root <none> and password file <none>
I20260812 19:05:49.163692 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:05:49.163885 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:05:49.164162 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:05:49.165047 28392 ts_manager.cc:194] Registered new tserver with Master: 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.165670 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-2-root/instance:
uuid: "0689ea7aff1049e28144361883e0ee37"
format_stamp: "Formatted at 2026-08-12 19:05:49 on dist-test-slave-vpvm"
I20260812 19:05:49.167174 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37176
I20260812 19:05:49.171514 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 19:05:49.174870 28732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.175570 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:05:49.175809 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
uuid: "0689ea7aff1049e28144361883e0ee37"
format_stamp: "Formatted at 2026-08-12 19:05:49 on dist-test-slave-vpvm"
I20260812 19:05:49.176054 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786561513794812-27445-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:05:49.187717 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:05:49.188798 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:05:49.190160 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:05:49.192252 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:05:49.192437 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.192655 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:05:49.192788 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:05:49.230849 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.67:37111
I20260812 19:05:49.230942 28830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.67:37111 every 8 connection(s)
I20260812 19:05:49.245250 28832 heartbeater.cc:347] Connected to a master server at 127.26.205.126:34053
I20260812 19:05:49.245672 28832 heartbeater.cc:464] Registering TS with master...
I20260812 19:05:49.246462 28832 heartbeater.cc:511] Master 127.26.205.126:34053 requested a full tablet report, sending...
I20260812 19:05:49.248311 28392 ts_manager.cc:194] Registered new tserver with Master: 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.248417 27445 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014552386s
I20260812 19:05:49.249995 28392 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37186
I20260812 19:05:49.250618 27445 test_util.cc:268] Using random seed: 1183002654
I20260812 19:05:49.274883 28392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37188:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:05:49.277838 28392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:05:49.360502 28540 tablet_service.cc:1467] Processing CreateTablet for tablet e41963dc3a804baebf22193068832e38 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260812 19:05:49.361040 28539 tablet_service.cc:1467] Processing CreateTablet for tablet 412149fd5f434de6a6f68f33580c7476 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260812 19:05:49.362262 28533 tablet_service.cc:1467] Processing CreateTablet for tablet dbcb053c08004c6db171442ca1a3b486 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260812 19:05:49.363080 28538 tablet_service.cc:1467] Processing CreateTablet for tablet 12ccdb81cc92432ebc3e4359ac0ccdca (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260812 19:05:49.363250 28530 tablet_service.cc:1467] Processing CreateTablet for tablet a038b5a28e9c4240a43afb3dc50c742c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260812 19:05:49.365409 28529 tablet_service.cc:1467] Processing CreateTablet for tablet e8ae7ad9bcb945869efb079d7719a015 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260812 19:05:49.367101 28535 tablet_service.cc:1467] Processing CreateTablet for tablet bfcd9c50dc4148f6be24136e2e58176c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260812 19:05:49.367265 28662 tablet_service.cc:1467] Processing CreateTablet for tablet e41963dc3a804baebf22193068832e38 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260812 19:05:49.367195 28779 tablet_service.cc:1467] Processing CreateTablet for tablet e41963dc3a804baebf22193068832e38 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260812 19:05:49.368613 28662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41963dc3a804baebf22193068832e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.368642 28779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41963dc3a804baebf22193068832e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.369131 28532 tablet_service.cc:1467] Processing CreateTablet for tablet f830c1407d4a41b693949b889e16cea2 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260812 19:05:49.369639 28769 tablet_service.cc:1467] Processing CreateTablet for tablet dbcb053c08004c6db171442ca1a3b486 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260812 19:05:49.370972 28769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbcb053c08004c6db171442ca1a3b486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.371079 28661 tablet_service.cc:1467] Processing CreateTablet for tablet 412149fd5f434de6a6f68f33580c7476 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260812 19:05:49.372421 28661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412149fd5f434de6a6f68f33580c7476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.372242 28650 tablet_service.cc:1467] Processing CreateTablet for tablet 80e6d885e328447ebd55ed5cf2690fe1 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260812 19:05:49.373080 28770 tablet_service.cc:1467] Processing CreateTablet for tablet bfcd9c50dc4148f6be24136e2e58176c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260812 19:05:49.374455 28646 tablet_service.cc:1467] Processing CreateTablet for tablet a038b5a28e9c4240a43afb3dc50c742c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260812 19:05:49.375104 28660 tablet_service.cc:1467] Processing CreateTablet for tablet 12ccdb81cc92432ebc3e4359ac0ccdca (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260812 19:05:49.375247 28772 tablet_service.cc:1467] Processing CreateTablet for tablet de8d6d7dce784fc0add3406e0413b541 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260812 19:05:49.377260 28643 tablet_service.cc:1467] Processing CreateTablet for tablet e8ae7ad9bcb945869efb079d7719a015 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260812 19:05:49.377393 28774 tablet_service.cc:1467] Processing CreateTablet for tablet 12ccdb81cc92432ebc3e4359ac0ccdca (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260812 19:05:49.379052 28656 tablet_service.cc:1467] Processing CreateTablet for tablet dbcb053c08004c6db171442ca1a3b486 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260812 19:05:49.379552 28777 tablet_service.cc:1467] Processing CreateTablet for tablet 412149fd5f434de6a6f68f33580c7476 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260812 19:05:49.381166 28531 tablet_service.cc:1467] Processing CreateTablet for tablet 80e6d885e328447ebd55ed5cf2690fe1 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260812 19:05:49.382542 28657 tablet_service.cc:1467] Processing CreateTablet for tablet bfcd9c50dc4148f6be24136e2e58176c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260812 19:05:49.377269 28658 tablet_service.cc:1467] Processing CreateTablet for tablet de8d6d7dce784fc0add3406e0413b541 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260812 19:05:49.385507 28650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e6d885e328447ebd55ed5cf2690fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.371097 28651 tablet_service.cc:1467] Processing CreateTablet for tablet f830c1407d4a41b693949b889e16cea2 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260812 19:05:49.386531 28777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412149fd5f434de6a6f68f33580c7476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.365335 28537 tablet_service.cc:1467] Processing CreateTablet for tablet de8d6d7dce784fc0add3406e0413b541 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260812 19:05:49.388924 28658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8d6d7dce784fc0add3406e0413b541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.389571 28646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a038b5a28e9c4240a43afb3dc50c742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.390815 28772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8d6d7dce784fc0add3406e0413b541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.392163 28660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ccdb81cc92432ebc3e4359ac0ccdca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.394380 28770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfcd9c50dc4148f6be24136e2e58176c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.395066 28540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e41963dc3a804baebf22193068832e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.398095 28529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ae7ad9bcb945869efb079d7719a015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.400353 28538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ccdb81cc92432ebc3e4359ac0ccdca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.394441 28656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbcb053c08004c6db171442ca1a3b486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.402141 28774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12ccdb81cc92432ebc3e4359ac0ccdca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.404363 28651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f830c1407d4a41b693949b889e16cea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.405078 28535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfcd9c50dc4148f6be24136e2e58176c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.405726 28539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412149fd5f434de6a6f68f33580c7476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.406576 28531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e6d885e328447ebd55ed5cf2690fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.407231 28530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a038b5a28e9c4240a43afb3dc50c742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.408681 28768 tablet_service.cc:1467] Processing CreateTablet for tablet f830c1407d4a41b693949b889e16cea2 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260812 19:05:49.410391 28532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f830c1407d4a41b693949b889e16cea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.411790 28767 tablet_service.cc:1467] Processing CreateTablet for tablet 80e6d885e328447ebd55ed5cf2690fe1 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260812 19:05:49.413235 28533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbcb053c08004c6db171442ca1a3b486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.414217 28657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfcd9c50dc4148f6be24136e2e58176c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.414897 28643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ae7ad9bcb945869efb079d7719a015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.416481 28537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de8d6d7dce784fc0add3406e0413b541. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.418274 28768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f830c1407d4a41b693949b889e16cea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.420359 28767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80e6d885e328447ebd55ed5cf2690fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.428033 28766 tablet_service.cc:1467] Processing CreateTablet for tablet a038b5a28e9c4240a43afb3dc50c742c (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260812 19:05:49.429451 28766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a038b5a28e9c4240a43afb3dc50c742c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.433367 28765 tablet_service.cc:1467] Processing CreateTablet for tablet e8ae7ad9bcb945869efb079d7719a015 (DEFAULT_TABLE table=test-workload [id=138f51e9240844e49f096634dd0bdfc4]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260812 19:05:49.434732 28765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8ae7ad9bcb945869efb079d7719a015. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:05:49.539280 28853 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.556907 28855 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.562086 28853 tablet_bootstrap.cc:654] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.562297 28854 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.563730 28855 tablet_bootstrap.cc:654] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.569293 28853 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.569940 28853 ts_tablet_manager.cc:1397] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.000s
I20260812 19:05:49.570675 28854 tablet_bootstrap.cc:654] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.573451 28853 raft_consensus.cc:348] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.574420 28853 raft_consensus.cc:374] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.574674 28853 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.575285 28854 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.575654 28854 ts_tablet_manager.cc:1397] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.002s
I20260812 19:05:49.575436 28853 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.576253 28855 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:49.576697 28855 ts_tablet_manager.cc:1397] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.000s
I20260812 19:05:49.577855 28832 heartbeater.cc:503] Master 127.26.205.126:34053 was elected leader, sending a full tablet report...
I20260812 19:05:49.578465 28853 ts_tablet_manager.cc:1428] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 19:05:49.579411 28853 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.579466 28855 raft_consensus.cc:348] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.580219 28855 raft_consensus.cc:374] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.580502 28855 raft_consensus.cc:729] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.581128 28854 raft_consensus.cc:348] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.581171 28855 consensus_queue.cc:260] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.581758 28854 raft_consensus.cc:374] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.582086 28854 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.582672 28854 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.584537 28855 ts_tablet_manager.cc:1428] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:05:49.585521 28855 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.590310 28712 heartbeater.cc:503] Master 127.26.205.126:34053 was elected leader, sending a full tablet report...
I20260812 19:05:49.591174 28596 heartbeater.cc:503] Master 127.26.205.126:34053 was elected leader, sending a full tablet report...
I20260812 19:05:49.591428 28854 ts_tablet_manager.cc:1428] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
I20260812 19:05:49.592190 28855 tablet_bootstrap.cc:654] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.592420 28854 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.598627 28854 tablet_bootstrap.cc:654] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.601069 28855 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:49.601541 28855 ts_tablet_manager.cc:1397] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20260812 19:05:49.604152 28853 tablet_bootstrap.cc:654] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.604202 28855 raft_consensus.cc:348] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.605100 28855 raft_consensus.cc:374] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.605404 28855 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.606258 28855 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.608705 28855 ts_tablet_manager.cc:1428] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260812 19:05:49.609627 28855 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.615661 28855 tablet_bootstrap.cc:654] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.620671 28861 raft_consensus.cc:482] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.621223 28861 raft_consensus.cc:504] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.631793 28861 leader_election.cc:288] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
W20260812 19:05:49.646293 28713 tablet.cc:2369] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:05:49.651017 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
W20260812 19:05:49.653151 28616 leader_election.cc:341] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.655135 28861 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.655630 28861 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.658520 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
W20260812 19:05:49.659925 28616 leader_election.cc:341] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.660866 28861 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.671195 28853 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.671682 28853 ts_tablet_manager.cc:1397] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.092s user 0.036s sys 0.034s
I20260812 19:05:49.672358 28854 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.672869 28854 ts_tablet_manager.cc:1397] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.081s user 0.021s sys 0.019s
I20260812 19:05:49.673161 28855 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:49.673611 28855 ts_tablet_manager.cc:1397] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.064s user 0.033s sys 0.028s
I20260812 19:05:49.675091 28853 raft_consensus.cc:348] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.675527 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:49.675856 28853 raft_consensus.cc:374] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.675665 28854 raft_consensus.cc:348] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.676113 28855 raft_consensus.cc:348] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.676752 28854 raft_consensus.cc:374] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.676883 28855 raft_consensus.cc:374] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.677064 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:49.677275 28855 raft_consensus.cc:729] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.677197 28854 raft_consensus.cc:729] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.676229 28853 raft_consensus.cc:729] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.678148 28855 consensus_queue.cc:260] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
W20260812 19:05:49.678328 28620 leader_election.cc:341] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.678402 28854 consensus_queue.cc:260] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.679045 28620 leader_election.cc:302] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 144c38de478a44859c4593c766e5b1da; no voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.678936 28853 consensus_queue.cc:260] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
W20260812 19:05:49.680260 28620 leader_election.cc:341] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.680593 28855 ts_tablet_manager.cc:1428] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:05:49.680653 28620 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 144c38de478a44859c4593c766e5b1da; no voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.681581 28855 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.682009 28853 ts_tablet_manager.cc:1428] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260812 19:05:49.683022 28853 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.680603 28854 ts_tablet_manager.cc:1428] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260812 19:05:49.684409 28854 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.687490 28855 tablet_bootstrap.cc:654] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.688951 28853 tablet_bootstrap.cc:654] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.691108 28871 raft_consensus.cc:2726] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.691277 28860 raft_consensus.cc:482] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.691742 28871 raft_consensus.cc:2726] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.691795 28860 raft_consensus.cc:504] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.710901 28860 leader_election.cc:288] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.716768 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
W20260812 19:05:49.718575 28735 leader_election.cc:341] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.691273 28854 tablet_bootstrap.cc:654] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.729202 28855 tablet_bootstrap.cc:492] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:49.729696 28855 ts_tablet_manager.cc:1397] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.048s user 0.017s sys 0.027s
I20260812 19:05:49.730094 28853 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.730566 28853 ts_tablet_manager.cc:1397] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.048s user 0.026s sys 0.019s
I20260812 19:05:49.730615 28854 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.732009 28855 raft_consensus.cc:348] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.732748 28854 ts_tablet_manager.cc:1397] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.012s
I20260812 19:05:49.733207 28853 raft_consensus.cc:348] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.733982 28853 raft_consensus.cc:374] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.734290 28853 raft_consensus.cc:729] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.735083 28853 consensus_queue.cc:260] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.735772 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.736366 28676 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:49.732722 28855 raft_consensus.cc:374] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.737161 28855 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.737552 28853 ts_tablet_manager.cc:1428] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:05:49.736069 28854 raft_consensus.cc:348] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.737737 28738 leader_election.cc:302] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters: 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.737994 28854 raft_consensus.cc:374] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.738485 28853 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.738448 28854 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.738910 28860 raft_consensus.cc:2781] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:49.740036 28860 raft_consensus.cc:482] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:49.739115 28855 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.740406 28860 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.740404 28854 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.743300 28854 ts_tablet_manager.cc:1428] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.010s user 0.004s sys 0.002s
I20260812 19:05:49.744289 28854 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.744725 28853 tablet_bootstrap.cc:654] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.747978 28860 raft_consensus.cc:504] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.750178 28861 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.750810 28854 tablet_bootstrap.cc:654] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.750623 28861 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.750824 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
W20260812 19:05:49.752259 28735 leader_election.cc:341] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.752599 28861 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.754474 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:49.755038 28676 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.755272 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:49.755834 28795 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:49.757541 28620 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:49.758294 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:49.758886 28558 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:49.764446 28861 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:49.764819 28861 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:49.765136 28861 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.765847 28676 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:49.769717 28860 leader_election.cc:288] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.771034 28860 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.771922 28860 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.773880 28860 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.774822 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:49.775498 28558 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:49.776705 28735 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:49.777567 28860 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:49.777894 28860 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:49.778232 28860 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.784103 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.785228 28738 leader_election.cc:302] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters: 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.786540 28880 raft_consensus.cc:2781] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:49.786398 28861 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.787551 28676 raft_consensus.cc:2379] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0689ea7aff1049e28144361883e0ee37 in current term 1: Already voted for candidate 144c38de478a44859c4593c766e5b1da in this term.
I20260812 19:05:49.789898 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:49.790515 28558 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.791584 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:49.792299 28861 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.793818 28862 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.794391 28862 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.796437 28855 ts_tablet_manager.cc:1428] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.066s user 0.004s sys 0.002s
I20260812 19:05:49.797156 28862 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.797405 28855 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.805919 28853 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.808447 28853 ts_tablet_manager.cc:1397] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.070s user 0.028s sys 0.028s
I20260812 19:05:49.811455 28853 raft_consensus.cc:348] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.812294 28853 raft_consensus.cc:374] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.812594 28853 raft_consensus.cc:729] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.807751 28558 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:49.816584 28855 tablet_bootstrap.cc:654] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.813338 28853 consensus_queue.cc:260] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.823575 28880 raft_consensus.cc:686] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:49.824559 28880 consensus_queue.cc:237] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.826218 28860 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.827674 28853 ts_tablet_manager.cc:1428] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260812 19:05:49.828317 28795 raft_consensus.cc:2379] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 144c38de478a44859c4593c766e5b1da in current term 1: Already voted for candidate 0689ea7aff1049e28144361883e0ee37 in this term.
I20260812 19:05:49.828645 28853 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.830193 28616 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:49.830830 28854 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.831303 28854 ts_tablet_manager.cc:1397] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.087s user 0.030s sys 0.046s
I20260812 19:05:49.831324 28861 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:49.831907 28861 raft_consensus.cc:686] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:49.832484 28860 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.832712 28861 consensus_queue.cc:237] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.833593 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:49.834360 28558 raft_consensus.cc:2379] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0689ea7aff1049e28144361883e0ee37 in current term 1: Already voted for candidate 144c38de478a44859c4593c766e5b1da in this term.
I20260812 19:05:49.834137 28854 raft_consensus.cc:348] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.834806 28854 raft_consensus.cc:374] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.835111 28854 raft_consensus.cc:729] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.836177 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:49.837694 28738 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37; no voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.838820 28860 raft_consensus.cc:2726] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.839531 28854 consensus_queue.cc:260] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.841777 28854 ts_tablet_manager.cc:1428] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260812 19:05:49.842612 28853 tablet_bootstrap.cc:654] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.842746 28854 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.848969 28854 tablet_bootstrap.cc:654] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.857110 28392 catalog_manager.cc:5495] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 0 to 1, leader changed from <none> to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 1 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:49.864935 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
W20260812 19:05:49.866251 28498 leader_election.cc:341] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.867955 28853 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.868506 28853 ts_tablet_manager.cc:1397] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.040s user 0.015s sys 0.012s
I20260812 19:05:49.870028 28854 tablet_bootstrap.cc:492] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.870632 28854 ts_tablet_manager.cc:1397] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.003s
I20260812 19:05:49.873432 28854 raft_consensus.cc:348] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.874205 28854 raft_consensus.cc:374] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.873999 28853 raft_consensus.cc:348] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.874516 28854 raft_consensus.cc:729] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.874751 28853 raft_consensus.cc:374] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.875049 28853 raft_consensus.cc:729] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.875299 28854 consensus_queue.cc:260] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.875749 28853 consensus_queue.cc:260] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.878253 28853 ts_tablet_manager.cc:1428] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.009s user 0.005s sys 0.001s
I20260812 19:05:49.879302 28853 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.879281 28391 catalog_manager.cc:5495] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 1 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
W20260812 19:05:49.882594 28471 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:05:49.883273 28862 raft_consensus.cc:482] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.883188 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.883895 28676 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 0.
I20260812 19:05:49.883770 28862 raft_consensus.cc:504] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.885291 28496 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:49.885704 28862 leader_election.cc:288] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.886236 28862 raft_consensus.cc:2781] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:49.886966 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:49.887610 28854 ts_tablet_manager.cc:1428] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260812 19:05:49.888572 28854 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.886619 28862 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:49.889371 28862 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.894649 28854 tablet_bootstrap.cc:654] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.895558 28855 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:49.896031 28855 ts_tablet_manager.cc:1397] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.099s user 0.000s sys 0.014s
I20260812 19:05:49.897012 28862 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.899163 28862 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.886221 28853 tablet_bootstrap.cc:654] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.900414 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:49.900946 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.901804 28676 raft_consensus.cc:2379] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e11a4f6072b426693a6664f95af0907 in current term 1: Already voted for candidate 0689ea7aff1049e28144361883e0ee37 in this term.
W20260812 19:05:49.902160 28498 leader_election.cc:341] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.903831 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:49.904466 28676 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.911801 28853 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.912395 28853 ts_tablet_manager.cc:1397] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.004s
I20260812 19:05:49.912427 28676 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 1.
I20260812 19:05:49.913763 28880 raft_consensus.cc:482] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.914371 28880 raft_consensus.cc:504] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.915512 28853 raft_consensus.cc:348] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.916353 28853 raft_consensus.cc:374] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.916694 28853 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.917694 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:49.917564 28853 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.919118 28496 leader_election.cc:302] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da
I20260812 19:05:49.920363 28853 ts_tablet_manager.cc:1428] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260812 19:05:49.920609 28862 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.920393 28880 leader_election.cc:288] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.921800 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
W20260812 19:05:49.923444 28738 leader_election.cc:341] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.924750 28853 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.928117 28862 raft_consensus.cc:2726] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.929363 28496 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
W20260812 19:05:49.930823 28735 leader_election.cc:341] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.931417 28853 tablet_bootstrap.cc:654] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.931304 28735 leader_election.cc:302] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37; no voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.932237 28860 raft_consensus.cc:2726] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.932987 28862 raft_consensus.cc:2781] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:49.933525 28862 raft_consensus.cc:686] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:49.934402 28862 consensus_queue.cc:237] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.938391 28901 raft_consensus.cc:482] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.938928 28901 raft_consensus.cc:504] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.941044 28901 leader_election.cc:288] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:49.942548 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
W20260812 19:05:49.944206 28498 leader_election.cc:341] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.944552 28855 raft_consensus.cc:348] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.945314 28855 raft_consensus.cc:374] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.945606 28855 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.947537 28880 raft_consensus.cc:482] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.947750 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.947960 28880 raft_consensus.cc:504] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
W20260812 19:05:49.948984 28496 leader_election.cc:341] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.949432 28496 leader_election.cc:302] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da
I20260812 19:05:49.949661 28880 leader_election.cc:288] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.950233 28901 raft_consensus.cc:2726] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.951216 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:49.951200 28855 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
W20260812 19:05:49.952831 28735 leader_election.cc:341] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.952970 28853 tablet_bootstrap.cc:492] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.953536 28853 ts_tablet_manager.cc:1397] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.000s
I20260812 19:05:49.951228 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.954761 28855 ts_tablet_manager.cc:1428] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.058s user 0.000s sys 0.008s
W20260812 19:05:49.955317 28738 leader_election.cc:341] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:49.955819 28855 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:49.955750 28738 leader_election.cc:302] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37; no voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.956563 28880 raft_consensus.cc:2726] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:49.956396 28853 raft_consensus.cc:348] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.957912 28853 raft_consensus.cc:374] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.958249 28853 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.959055 28853 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.961592 28854 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.961627 28853 ts_tablet_manager.cc:1428] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260812 19:05:49.962299 28854 ts_tablet_manager.cc:1397] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.074s user 0.021s sys 0.035s
I20260812 19:05:49.962817 28853 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.963732 28855 tablet_bootstrap.cc:654] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.965086 28854 raft_consensus.cc:348] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.965814 28854 raft_consensus.cc:374] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.966138 28854 raft_consensus.cc:729] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.966849 28854 consensus_queue.cc:260] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.969061 28853 tablet_bootstrap.cc:654] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.969405 28854 ts_tablet_manager.cc:1428] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260812 19:05:49.970490 28854 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.975111 28853 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:49.975574 28853 ts_tablet_manager.cc:1397] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 19:05:49.976583 28854 tablet_bootstrap.cc:654] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.979802 28853 raft_consensus.cc:348] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.980535 28853 raft_consensus.cc:374] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.980808 28853 raft_consensus.cc:729] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.981562 28853 consensus_queue.cc:260] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.983158 28853 ts_tablet_manager.cc:1428] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 19:05:49.983502 28854 tablet_bootstrap.cc:492] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:49.983805 28853 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:49.984114 28854 ts_tablet_manager.cc:1397] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260812 19:05:49.986968 28854 raft_consensus.cc:348] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.987812 28854 raft_consensus.cc:374] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:49.988128 28880 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.988174 28854 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:49.988754 28880 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.989291 28854 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:49.990852 28880 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:49.991638 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:49.991851 28854 ts_tablet_manager.cc:1428] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260812 19:05:49.992419 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:49.992969 28854 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:49.993113 28676 raft_consensus.cc:2379] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0689ea7aff1049e28144361883e0ee37 in current term 1: Already voted for candidate 4e11a4f6072b426693a6664f95af0907 in this term.
I20260812 19:05:49.989415 28391 catalog_manager.cc:5495] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:49.994837 28735 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37; no voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:49.996937 28880 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:49.997416 28853 tablet_bootstrap.cc:654] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.999236 28860 raft_consensus.cc:482] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:49.999377 28854 tablet_bootstrap.cc:654] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:49.999686 28860 raft_consensus.cc:504] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.001685 28860 leader_election.cc:288] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.003101 28861 raft_consensus.cc:482] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.003262 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:50.003629 28861 raft_consensus.cc:504] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.004845 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.005475 28558 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:50.005684 28861 leader_election.cc:288] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.007225 28735 leader_election.cc:302] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.007813 28862 raft_consensus.cc:482] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.008328 28862 raft_consensus.cc:504] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.009267 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:50.009903 28795 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:50.010336 28862 leader_election.cc:288] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.011112 28620 leader_election.cc:302] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:50.007628 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.012300 28854 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:50.013046 28854 ts_tablet_manager.cc:1397] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.009s
I20260812 19:05:50.013466 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
W20260812 19:05:50.013633 28738 leader_election.cc:341] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:50.014528 28860 raft_consensus.cc:2781] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.014878 28860 raft_consensus.cc:482] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:05:50.014921 28498 leader_election.cc:341] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:50.015322 28860 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.012436 28558 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:50.016207 28854 raft_consensus.cc:348] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.017002 28854 raft_consensus.cc:374] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.017298 28854 raft_consensus.cc:729] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.012343 28880 raft_consensus.cc:2726] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:50.012142 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:50.018136 28861 raft_consensus.cc:2781] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.018640 28675 raft_consensus.cc:2454] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 0.
I20260812 19:05:50.018838 28855 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:50.018004 28854 consensus_queue.cc:260] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.019802 28496 leader_election.cc:302] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:50.020812 28854 ts_tablet_manager.cc:1428] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260812 19:05:50.018548 28861 raft_consensus.cc:482] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.021541 28861 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.021867 28854 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:50.021111 28862 raft_consensus.cc:2781] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.024408 28862 raft_consensus.cc:482] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.024757 28862 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.027215 28854 tablet_bootstrap.cc:654] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.028859 28861 raft_consensus.cc:504] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.032008 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.032667 28558 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.036060 28862 raft_consensus.cc:504] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.039177 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.039817 28675 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.044998 28860 raft_consensus.cc:504] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.046231 28855 ts_tablet_manager.cc:1397] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.091s user 0.010s sys 0.008s
I20260812 19:05:50.048593 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.049404 28557 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.039697 28558 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.051910 28616 leader_election.cc:302] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.052971 28871 raft_consensus.cc:2781] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.049302 28855 raft_consensus.cc:348] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.060940 28862 leader_election.cc:288] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.055540 28855 raft_consensus.cc:374] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.061928 28855 raft_consensus.cc:729] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.062417 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.062907 28855 consensus_queue.cc:260] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.065196 28871 raft_consensus.cc:686] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
W20260812 19:05:50.064441 28498 leader_election.cc:341] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:50.066898 28871 consensus_queue.cc:237] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.070322 28860 leader_election.cc:288] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.070720 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.071005 28861 leader_election.cc:288] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
W20260812 19:05:50.072556 28738 leader_election.cc:341] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:50.072850 28675 raft_consensus.cc:2454] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 1.
I20260812 19:05:50.073721 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.078280 28496 leader_election.cc:302] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:50.079806 28795 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.082183 28862 raft_consensus.cc:2781] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.082729 28862 raft_consensus.cc:686] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.083632 28862 consensus_queue.cc:237] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.086433 28557 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:50.087721 28735 leader_election.cc:302] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters: 144c38de478a44859c4593c766e5b1da
I20260812 19:05:50.089401 28860 raft_consensus.cc:2781] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.089691 28795 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.090092 28860 raft_consensus.cc:686] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.091128 28860 consensus_queue.cc:237] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.096235 28861 raft_consensus.cc:482] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.096773 28861 raft_consensus.cc:504] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.098842 28853 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:50.098996 28861 leader_election.cc:288] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.099284 28854 tablet_bootstrap.cc:492] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:50.099424 28853 ts_tablet_manager.cc:1397] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.116s user 0.043s sys 0.028s
I20260812 19:05:50.101814 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.102823 28854 ts_tablet_manager.cc:1397] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.081s user 0.032s sys 0.032s
W20260812 19:05:50.103570 28616 leader_election.cc:341] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:05:50.104624 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:50.102129 28853 raft_consensus.cc:348] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.105337 28795 raft_consensus.cc:2454] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:50.106777 28853 raft_consensus.cc:374] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.107338 28620 leader_election.cc:302] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters: 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.107517 28853 raft_consensus.cc:729] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.108490 28861 raft_consensus.cc:2781] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.108857 28861 raft_consensus.cc:482] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.109210 28861 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.108681 28853 consensus_queue.cc:260] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.105436 28854 raft_consensus.cc:348] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.111258 28854 raft_consensus.cc:374] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.111547 28854 raft_consensus.cc:729] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.112166 28853 ts_tablet_manager.cc:1428] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20260812 19:05:50.112166 28854 consensus_queue.cc:260] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.111860 28390 catalog_manager.cc:5495] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 1 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:50.113366 28853 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: Bootstrap starting.
I20260812 19:05:50.114570 28854 ts_tablet_manager.cc:1428] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.011s user 0.005s sys 0.001s
I20260812 19:05:50.115422 28854 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: Bootstrap starting.
I20260812 19:05:50.116164 28391 catalog_manager.cc:5495] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.116940 28392 catalog_manager.cc:5495] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 0 to 1, leader changed from <none> to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 1 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.119418 28853 tablet_bootstrap.cc:654] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.120501 28854 tablet_bootstrap.cc:654] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.128858 28861 raft_consensus.cc:504] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.136407 28855 ts_tablet_manager.cc:1428] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.090s user 0.005s sys 0.023s
I20260812 19:05:50.137501 28855 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:50.152689 28854 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: No bootstrap required, opened a new log
I20260812 19:05:50.153141 28854 ts_tablet_manager.cc:1397] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.020s
I20260812 19:05:50.151751 28861 leader_election.cc:288] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.153653 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.154297 28557 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.155380 28854 raft_consensus.cc:348] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.156057 28854 raft_consensus.cc:374] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.155936 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.156348 28854 raft_consensus.cc:729] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.156560 28795 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.156962 28854 consensus_queue.cc:260] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.159240 28854 ts_tablet_manager.cc:1428] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260812 19:05:50.162096 28557 raft_consensus.cc:2454] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.163434 28616 leader_election.cc:302] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.164224 28861 raft_consensus.cc:2781] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.164765 28861 raft_consensus.cc:686] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.165665 28861 consensus_queue.cc:237] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.170466 28795 raft_consensus.cc:2454] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.172801 28853 tablet_bootstrap.cc:492] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: No bootstrap required, opened a new log
I20260812 19:05:50.173312 28853 ts_tablet_manager.cc:1397] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: Time spent bootstrapping tablet: real 0.060s user 0.036s sys 0.016s
I20260812 19:05:50.176007 28853 raft_consensus.cc:348] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.176689 28853 raft_consensus.cc:374] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.170833 28855 tablet_bootstrap.cc:654] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.176980 28853 raft_consensus.cc:729] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.177600 28853 consensus_queue.cc:260] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.179661 28853 ts_tablet_manager.cc:1428] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 19:05:50.179663 28390 catalog_manager.cc:5495] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 0 to 1, leader changed from <none> to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 1 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.182456 28855 tablet_bootstrap.cc:492] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:50.182931 28855 ts_tablet_manager.cc:1397] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.000s
I20260812 19:05:50.185045 28855 raft_consensus.cc:348] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.185755 28855 raft_consensus.cc:374] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.186095 28855 raft_consensus.cc:729] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.186784 28855 consensus_queue.cc:260] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.189052 28855 ts_tablet_manager.cc:1428] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:05:50.190045 28855 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:50.195178 28855 tablet_bootstrap.cc:654] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.199117 28855 tablet_bootstrap.cc:492] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:50.199461 28855 ts_tablet_manager.cc:1397] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260812 19:05:50.201092 28855 raft_consensus.cc:348] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.201555 28855 raft_consensus.cc:374] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.201711 28855 raft_consensus.cc:729] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.202168 28855 consensus_queue.cc:260] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.203457 28855 ts_tablet_manager.cc:1428] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 19:05:50.204391 28855 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:50.208408 28855 tablet_bootstrap.cc:654] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.212591 28855 tablet_bootstrap.cc:492] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:50.212987 28855 ts_tablet_manager.cc:1397] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 19:05:50.213589 28861 raft_consensus.cc:482] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.213977 28861 raft_consensus.cc:504] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.214836 28855 raft_consensus.cc:348] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.215339 28855 raft_consensus.cc:374] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.215545 28855 raft_consensus.cc:729] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.215564 28861 leader_election.cc:288] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.216658 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.216372 28855 consensus_queue.cc:260] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.216708 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:50.217326 28557 raft_consensus.cc:2379] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 144c38de478a44859c4593c766e5b1da in current term 1: Already voted for candidate 0689ea7aff1049e28144361883e0ee37 in this term.
I20260812 19:05:50.218351 28855 ts_tablet_manager.cc:1428] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 19:05:50.219110 28616 leader_election.cc:302] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 144c38de478a44859c4593c766e5b1da; no voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.219298 28855 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: Bootstrap starting.
I20260812 19:05:50.219758 28861 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.224915 28855 tablet_bootstrap.cc:654] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: Neither blocks nor log segments found. Creating new log.
I20260812 19:05:50.225737 28861 raft_consensus.cc:2726] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:50.229144 28855 tablet_bootstrap.cc:492] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: No bootstrap required, opened a new log
I20260812 19:05:50.229506 28855 ts_tablet_manager.cc:1397] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260812 19:05:50.231480 28855 raft_consensus.cc:348] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.232030 28855 raft_consensus.cc:374] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:05:50.232020 28861 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.232419 28855 raft_consensus.cc:729] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Initialized, Role: FOLLOWER
I20260812 19:05:50.232918 28855 consensus_queue.cc:260] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.234491 28855 ts_tablet_manager.cc:1428] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260812 19:05:50.249045 28861 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.300755 28860 raft_consensus.cc:482] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.301177 28860 raft_consensus.cc:504] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.302876 28860 leader_election.cc:288] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.303696 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.303881 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:50.304381 28557 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:50.304474 28675 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:50.305428 28738 leader_election.cc:302] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:50.306051 28860 raft_consensus.cc:2781] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.306394 28860 raft_consensus.cc:482] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.306643 28860 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.311120 28860 raft_consensus.cc:504] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.312568 28860 leader_election.cc:288] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.313328 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.313534 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.313813 28557 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.314052 28675 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.318207 28675 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:50.318216 28557 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:50.319136 28738 leader_election.cc:302] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:50.319746 28860 raft_consensus.cc:2781] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.320142 28860 raft_consensus.cc:686] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.320835 28860 consensus_queue.cc:237] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.328083 28389 catalog_manager.cc:5495] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 1 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:50.339108 28925 raft_consensus.cc:482] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.339548 28925 raft_consensus.cc:504] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.341104 28925 leader_election.cc:288] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.341943 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.342109 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37" is_pre_election: true
I20260812 19:05:50.342528 28557 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:50.342641 28795 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 0.
I20260812 19:05:50.343385 28616 leader_election.cc:302] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.344046 28925 raft_consensus.cc:2781] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.344290 28925 raft_consensus.cc:482] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.344497 28925 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.348973 28925 raft_consensus.cc:504] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.349386 28860 raft_consensus.cc:482] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.349831 28860 raft_consensus.cc:504] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.350831 28925 leader_election.cc:288] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.351701 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.351877 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.352396 28557 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.352458 28795 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.353472 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.353873 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:50.354628 28675 raft_consensus.cc:2379] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0689ea7aff1049e28144361883e0ee37 in current term 1: Already voted for candidate 4e11a4f6072b426693a6664f95af0907 in this term.
I20260812 19:05:50.355754 28738 leader_election.cc:302] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37; no voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.355934 28862 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.352437 28860 leader_election.cc:288] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.356956 28860 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.359764 28795 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.360809 28620 leader_election.cc:302] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:50.361624 28925 raft_consensus.cc:2781] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.362010 28557 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 1.
I20260812 19:05:50.362106 28925 raft_consensus.cc:686] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.363157 28925 consensus_queue.cc:237] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.363709 28860 raft_consensus.cc:2726] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:05:50.372454 28390 catalog_manager.cc:5495] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 0 to 1, leader changed from <none> to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 1 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.374089 28901 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.415212 28860 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.420735 28860 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.469521 28960 raft_consensus.cc:482] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:05:50.470067 28960 raft_consensus.cc:504] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.471808 28960 leader_election.cc:288] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.472673 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907" is_pre_election: true
I20260812 19:05:50.472780 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da" is_pre_election: true
I20260812 19:05:50.473227 28557 raft_consensus.cc:2454] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:50.473294 28675 raft_consensus.cc:2454] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 0.
I20260812 19:05:50.474341 28735 leader_election.cc:302] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.474967 28960 raft_consensus.cc:2781] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:05:50.475286 28960 raft_consensus.cc:482] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:05:50.475510 28960 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.480175 28960 raft_consensus.cc:504] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.481673 28960 leader_election.cc:288] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:50.482627 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.482686 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.483114 28557 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.483176 28675 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:05:50.488174 28675 raft_consensus.cc:2454] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:50.488299 28557 raft_consensus.cc:2454] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 1.
I20260812 19:05:50.489189 28735 leader_election.cc:302] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.489080 28906 consensus_queue.cc:1035] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:50.489838 28960 raft_consensus.cc:2781] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:05:50.490320 28960 raft_consensus.cc:686] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.491044 28960 consensus_queue.cc:237] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.500334 28389 catalog_manager.cc:5495] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 1 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:50.502373 28901 consensus_queue.cc:1035] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.533869 28675 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:05:50.534497 28675 raft_consensus.cc:595] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.539245 28675 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:05:50.539742 28675 raft_consensus.cc:595] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.554044 28798 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:05:50.554615 28798 raft_consensus.cc:595] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.557371 28925 consensus_queue.cc:1035] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.559583 28675 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:05:50.560184 28675 raft_consensus.cc:595] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.564134 28861 raft_consensus.cc:979] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.564504 28861 raft_consensus.cc:1067] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.565977 28798 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:05:50.566555 28798 raft_consensus.cc:595] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:05:50.567442 28558 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:05:50.568332 28558 raft_consensus.cc:482] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.568725 28558 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.569978 28871 consensus_queue.cc:1035] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.576131 28558 raft_consensus.cc:504] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.579133 28558 leader_election.cc:288] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.580866 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.582974 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de8d6d7dce784fc0add3406e0413b541" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.583874 28675 raft_consensus.cc:3032] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.584199 28675 raft_consensus.cc:729] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.586863 28960 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.587265 28675 consensus_queue.cc:260] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.588416 28675 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.593367 28960 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.594985 28675 raft_consensus.cc:2454] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.596184 28496 leader_election.cc:302] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters: 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:50.597271 28901 raft_consensus.cc:2781] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.597733 28901 raft_consensus.cc:686] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.598664 28901 consensus_queue.cc:237] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.616309 28557 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:47600
I20260812 19:05:50.616925 28557 raft_consensus.cc:595] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Received request to transfer leadership to TS 144c38de478a44859c4593c766e5b1da
I20260812 19:05:50.619124 28389 catalog_manager.cc:5495] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.623005 28798 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:05:50.623636 28798 raft_consensus.cc:595] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Received request to transfer leadership to TS 144c38de478a44859c4593c766e5b1da
I20260812 19:05:50.623711 28861 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.630488 28860 raft_consensus.cc:979] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.630894 28880 raft_consensus.cc:1067] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Signalling peer 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.631821 28871 raft_consensus.cc:979] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.632225 28871 raft_consensus.cc:1067] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.636143 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:05:50.636675 28557 raft_consensus.cc:482] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.637045 28557 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.639716 28925 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.642145 28795 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:05:50.642680 28795 raft_consensus.cc:595] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Received request to transfer leadership to TS 144c38de478a44859c4593c766e5b1da
I20260812 19:05:50.646188 28676 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:37452
I20260812 19:05:50.646799 28676 raft_consensus.cc:482] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.647226 28676 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.649374 28557 raft_consensus.cc:504] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.649629 28558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:47600
I20260812 19:05:50.650379 28558 raft_consensus.cc:595] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Received request to transfer leadership to TS 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:50.654063 28676 raft_consensus.cc:504] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.655264 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.655858 28675 raft_consensus.cc:3032] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.656173 28675 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.657231 28676 leader_election.cc:288] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.657881 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.658401 28795 raft_consensus.cc:3032] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.658737 28795 raft_consensus.cc:729] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.659397 28795 consensus_queue.cc:260] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.660346 28795 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.656886 28675 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.663347 28557 leader_election.cc:288] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.663225 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.663784 28675 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.663861 28558 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.664297 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.664880 28798 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.668876 28558 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.669893 28616 leader_election.cc:302] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.670387 28675 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.670590 28871 raft_consensus.cc:2781] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.671132 28871 raft_consensus.cc:686] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.671502 28496 leader_election.cc:302] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.671900 28871 consensus_queue.cc:237] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.672350 28901 raft_consensus.cc:2781] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.672917 28901 raft_consensus.cc:686] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.673754 28901 consensus_queue.cc:237] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.677929 28798 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.679436 28390 catalog_manager.cc:5495] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 1 to 2, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 2 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.680459 28795 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.682268 28392 catalog_manager.cc:5495] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.703490 28925 raft_consensus.cc:979] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.703965 28871 raft_consensus.cc:1067] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.704228 28880 consensus_queue.cc:1035] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.705588 28558 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:05:50.706068 28558 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.706449 28558 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.711331 28860 raft_consensus.cc:979] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.711810 28860 raft_consensus.cc:1067] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Signalling peer 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.712721 28558 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.715425 28558 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.717525 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.717489 28674 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:37452
I20260812 19:05:50.718197 28795 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.718344 28674 raft_consensus.cc:482] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.718752 28674 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.720300 28960 consensus_queue.cc:1035] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.724068 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.724658 28675 raft_consensus.cc:3032] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.724977 28675 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.725936 28795 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.725785 28675 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.727056 28675 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.727128 28498 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.728096 28901 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.728701 28901 raft_consensus.cc:686] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.729826 28901 consensus_queue.cc:237] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.730055 28674 raft_consensus.cc:504] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.732451 28674 leader_election.cc:288] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.733304 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.733858 28557 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.734416 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a038b5a28e9c4240a43afb3dc50c742c" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.734948 28795 raft_consensus.cc:3032] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.735257 28795 raft_consensus.cc:729] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.735977 28795 consensus_queue.cc:260] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.737123 28795 raft_consensus.cc:3037] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.740285 28675 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.743240 28557 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.744791 28795 raft_consensus.cc:2454] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.745877 28616 leader_election.cc:302] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.747263 28871 raft_consensus.cc:2781] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.747641 28871 raft_consensus.cc:686] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.748306 28871 consensus_queue.cc:237] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.748322 28390 catalog_manager.cc:5495] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.755610 28392 catalog_manager.cc:5495] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 1 to 2, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 2 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.797891 28871 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.805852 28960 raft_consensus.cc:979] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.806322 28960 raft_consensus.cc:1067] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.809883 28558 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47596
I20260812 19:05:50.810467 28558 raft_consensus.cc:482] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.810825 28558 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.812570 28927 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.818648 28558 raft_consensus.cc:504] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.820914 28558 leader_election.cc:288] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.821533 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.821797 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.822144 28795 raft_consensus.cc:3032] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.822346 28675 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.822517 28795 raft_consensus.cc:729] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.823431 28795 consensus_queue.cc:260] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.824649 28795 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.829066 28675 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.830166 28496 leader_election.cc:302] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:05:50.830801 28901 raft_consensus.cc:2781] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.830710 28795 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.831300 28901 raft_consensus.cc:686] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.832090 28901 consensus_queue.cc:237] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.837976 28392 catalog_manager.cc:5495] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.873793 28860 consensus_queue.cc:1035] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:05:50.881307 28880 raft_consensus.cc:979] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.881744 28880 raft_consensus.cc:1067] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:05:50.885771 28960 consensus_queue.cc:1035] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.886291 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47596
I20260812 19:05:50.886814 28557 raft_consensus.cc:482] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.887178 28557 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.888897 28906 raft_consensus.cc:979] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.889284 28901 raft_consensus.cc:1067] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Signalling peer 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:05:50.890793 28675 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:37466
I20260812 19:05:50.891373 28675 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:50.891741 28675 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.895613 28557 raft_consensus.cc:504] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.896864 28558 raft_consensus.cc:1226] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Rejecting Update request from peer 0689ea7aff1049e28144361883e0ee37 for earlier term 1. Current term is 2. Ops: []
I20260812 19:05:50.898166 28675 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.898548 28676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:50.900786 28675 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.900610 28880 consensus_queue.cc:1046] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:05:50.901048 28557 leader_election.cc:288] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:05:50.901628 28880 raft_consensus.cc:3032] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.902315 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12ccdb81cc92432ebc3e4359ac0ccdca" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.902701 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:05:50.903007 28880 raft_consensus.cc:729] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:50.903337 28795 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.904085 28880 consensus_queue.cc:260] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.905224 28880 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.910038 28795 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.911312 28620 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:50.912354 28927 raft_consensus.cc:2781] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.912817 28927 raft_consensus.cc:686] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:05:50.913702 28927 consensus_queue.cc:237] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.917846 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:50.918490 28558 raft_consensus.cc:3032] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:50.918810 28558 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.919644 28558 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.920866 28558 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:50.923055 28798 raft_consensus.cc:2454] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:05:50.924780 28498 leader_election.cc:302] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters: 144c38de478a44859c4593c766e5b1da
I20260812 19:05:50.925626 28862 raft_consensus.cc:2781] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:50.926131 28862 raft_consensus.cc:686] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:50.926950 28558 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 2.
I20260812 19:05:50.927214 28862 consensus_queue.cc:237] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:50.927321 28390 catalog_manager.cc:5495] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 1 to 2, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 2 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:50.940126 28392 catalog_manager.cc:5495] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:05:51.048108 28901 raft_consensus.cc:979] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:51.048561 28901 raft_consensus.cc:1067] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Signalling peer 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:51.050061 28798 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:51890
I20260812 19:05:51.050520 28798 raft_consensus.cc:482] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:51.050990 28798 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:51.056217 28798 raft_consensus.cc:504] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:51.057777 28798 leader_election.cc:288] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 2 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:51.059052 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:51.059124 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8ae7ad9bcb945869efb079d7719a015" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:51.059612 28674 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:51.059803 28558 raft_consensus.cc:3032] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:05:51.059959 28557 raft_consensus.cc:1261] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.060196 28558 raft_consensus.cc:729] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:51.060976 28558 consensus_queue.cc:260] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:51.061354 28927 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.062106 28558 raft_consensus.cc:3037] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:51.065371 28674 raft_consensus.cc:2454] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 2.
I20260812 19:05:51.066409 28738 leader_election.cc:302] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:51.069334 28798 raft_consensus.cc:1261] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.070807 28871 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.071848 28880 raft_consensus.cc:2781] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:05:51.073149 28798 raft_consensus.cc:3037] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:51.073216 28880 raft_consensus.cc:686] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 2 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:51.074349 28880 consensus_queue.cc:237] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:51.077795 28558 raft_consensus.cc:2454] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 2.
I20260812 19:05:51.080224 28798 raft_consensus.cc:1261] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.081809 28901 consensus_queue.cc:1035] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.083724 28392 catalog_manager.cc:5495] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 1 to 2, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 2 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:51.089986 28674 raft_consensus.cc:1261] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.092778 28906 consensus_queue.cc:1035] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.199577 28674 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.200822 28862 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.208455 28798 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.210299 28901 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.227229 28558 raft_consensus.cc:1261] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.228374 28927 consensus_queue.cc:1035] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.235363 28798 raft_consensus.cc:1261] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.235443 28795 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.237012 28871 consensus_queue.cc:1035] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.237509 28901 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.246268 28674 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.247640 28901 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.354508 28798 raft_consensus.cc:1261] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.355681 28901 consensus_queue.cc:1035] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.363107 28674 raft_consensus.cc:3037] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:05:51.365178 28798 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.366616 28871 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.375690 28558 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.377110 28925 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.377398 28675 raft_consensus.cc:1261] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.379594 28862 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.382440 28674 raft_consensus.cc:1261] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.384809 28901 consensus_queue.cc:1035] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.393215 28798 raft_consensus.cc:1261] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.394871 28991 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:05:51.502514 28558 raft_consensus.cc:1261] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.503808 28960 consensus_queue.cc:1035] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:51.511917 28675 raft_consensus.cc:1261] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:05:51.513767 28860 consensus_queue.cc:1035] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:05:53.678845 27445 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20260812 19:05:53.679478 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:53.696436 28558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:05:53.696942 28558 raft_consensus.cc:595] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Received request to transfer leadership to TS 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:53.699045 28558 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:05:53.699498 28558 raft_consensus.cc:595] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Received request to transfer leadership to TS 0689ea7aff1049e28144361883e0ee37
I20260812 19:05:53.700354 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260812 19:05:53.700763 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:53.847908 28862 raft_consensus.cc:979] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:53.848316 29000 raft_consensus.cc:1067] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Signalling peer 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:53.849736 28795 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:51890
I20260812 19:05:53.850226 28795 raft_consensus.cc:482] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:53.850495 28795 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:53.855088 28795 raft_consensus.cc:504] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:53.856630 28795 leader_election.cc:288] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:53.857640 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:53.857671 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:53.858245 28558 raft_consensus.cc:3032] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:53.858296 28675 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:53.858538 28558 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:53.859223 28558 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:53.860175 28558 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:53.863005 28675 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:05:53.864030 28738 leader_election.cc:302] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:53.864696 28860 raft_consensus.cc:2781] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:53.864782 28558 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:05:53.865871 28860 raft_consensus.cc:686] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:53.866648 28860 consensus_queue.cc:237] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:53.873430 28392 catalog_manager.cc:5495] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 2 to 3, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 3 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:54.208916 28862 raft_consensus.cc:979] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:54.209661 29000 raft_consensus.cc:1067] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Signalling peer 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:05:54.211329 28795 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:51890
I20260812 19:05:54.211982 28795 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:05:54.212463 28795 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:54.221150 28795 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:54.223097 28795 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:05:54.224675 28558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:05:54.224661 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:05:54.225591 28558 raft_consensus.cc:3032] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:05:54.225723 28675 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:54.225996 28558 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:05:54.226907 28558 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:54.228513 28558 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:05:54.234663 28675 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:05:54.235994 28738 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:05:54.236919 28558 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:05:54.237062 28860 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:05:54.237542 28860 raft_consensus.cc:686] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:05:54.239096 28860 consensus_queue.cc:237] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:05:54.257997 28392 catalog_manager.cc:5495] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 2 to 3, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 3 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:05:54.260531 28675 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:54.262048 28860 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:54.278196 28558 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:54.280983 29009 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:05:54.701908 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:54.707582 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:54.707985 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:54.731690 28558 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:54.732863 29009 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:54.740809 28674 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:05:54.742185 29009 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:05:55.708822 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:55.714059 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:55.714483 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:56.715298 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:56.720695 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:56.721092 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:57.722012 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:57.727252 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:57.727681 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:58.728617 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:58.734869 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:58.735239 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:05:59.735985 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:05:59.742385 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:05:59.742738 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:00.743510 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:00.748574 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:00.748932 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:01.749735 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:01.755093 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:01.755471 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:02.756234 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:02.761981 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:02.762408 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:03.792745 27445 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20260812 19:06:03.796785 28798 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:06:03.797293 28798 raft_consensus.cc:595] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:06:03.801213 28798 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:51900
I20260812 19:06:03.801617 28798 raft_consensus.cc:595] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:06:03.811985 28674 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:06:03.812443 28674 raft_consensus.cc:595] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:06:03.816481 28674 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:06:03.816912 28674 raft_consensus.cc:595] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:06:03.820710 28674 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
mode: GRACEFUL
new_leader_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:37468
I20260812 19:06:03.821113 28674 raft_consensus.cc:595] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Received request to transfer leadership to TS 4e11a4f6072b426693a6664f95af0907
I20260812 19:06:03.843636 29009 raft_consensus.cc:979] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.844086 29037 raft_consensus.cc:1067] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.845482 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47596
I20260812 19:06:03.845942 28557 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:03.846258 28557 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.851004 28557 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.852651 28557 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 4 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:03.853461 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:03.853708 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:03.854075 28674 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.854280 28798 raft_consensus.cc:3032] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:06:03.854591 28798 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:06:03.855206 28798 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.856226 28798 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.858721 29019 raft_consensus.cc:979] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.859315 29038 raft_consensus.cc:1067] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.860948 28674 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 4.
I20260812 19:06:03.860999 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:06:03.861615 28557 raft_consensus.cc:482] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:03.861732 28798 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 4.
I20260812 19:06:03.862130 28557 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:03.862457 28496 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:03.863202 29000 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:06:03.864112 29000 raft_consensus.cc:686] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:03.864995 29000 consensus_queue.cc:237] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.868510 28557 raft_consensus.cc:504] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.870462 28557 leader_election.cc:288] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:03.871147 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:03.871688 28674 raft_consensus.cc:3032] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:03.871906 28674 raft_consensus.cc:729] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:06:03.872481 28674 consensus_queue.cc:260] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.872102 28392 catalog_manager.cc:5495] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 3 to 4, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:03.873276 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:03.873593 28674 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:03.873811 28798 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:03.879649 28798 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:06:03.880769 28498 leader_election.cc:302] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:03.881667 29000 raft_consensus.cc:2781] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:03.881723 28674 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 2.
I20260812 19:06:03.882285 29000 raft_consensus.cc:686] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:03.882992 29000 consensus_queue.cc:237] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.891139 28392 catalog_manager.cc:5495] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 1 to 2, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:03.982093 29009 raft_consensus.cc:979] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.982487 29037 raft_consensus.cc:1067] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.983722 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47596
I20260812 19:06:03.984165 28557 raft_consensus.cc:482] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:03.984431 28557 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.988874 28557 raft_consensus.cc:504] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.990370 28557 leader_election.cc:288] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 4 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:03.991035 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:03.991230 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412149fd5f434de6a6f68f33580c7476" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:03.991600 28674 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.991714 28798 raft_consensus.cc:3032] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:06:03.992038 28798 raft_consensus.cc:729] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:06:03.992712 28798 consensus_queue.cc:260] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:03.993773 28798 raft_consensus.cc:3037] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:03.995748 29019 raft_consensus.cc:979] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.996191 29038 raft_consensus.cc:1067] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:03.996816 28674 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 4.
I20260812 19:06:03.997599 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:06:03.997736 28496 leader_election.cc:302] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144c38de478a44859c4593c766e5b1da, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:03.998159 28557 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:03.998512 28557 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:03.998898 29000 raft_consensus.cc:2781] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:06:03.998953 28798 raft_consensus.cc:2454] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 4.
I20260812 19:06:03.999459 29000 raft_consensus.cc:686] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:04.000259 29000 consensus_queue.cc:237] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.004398 28557 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.006227 28557 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 3 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:04.006788 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:04.006850 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:04.007426 28798 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:04.007704 28674 raft_consensus.cc:3032] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:06:04.007983 28674 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:06:04.007902 28392 catalog_manager.cc:5495] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 3 to 4, leader changed from 0689ea7aff1049e28144361883e0ee37 (127.26.205.67) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:04.008600 28674 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.009534 28674 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:04.012463 28798 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 3.
I20260812 19:06:04.013408 28498 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:04.014199 29000 raft_consensus.cc:2781] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:04.014608 29000 raft_consensus.cc:686] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:04.015027 28674 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 3.
I20260812 19:06:04.015308 29000 consensus_queue.cc:237] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.022869 28392 catalog_manager.cc:5495] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 2 to 3, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:04.158767 29019 raft_consensus.cc:979] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: : Instructing follower 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:04.159198 29038 raft_consensus.cc:1067] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Signalling peer 4e11a4f6072b426693a6664f95af0907 to start an election
I20260812 19:06:04.160527 28557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
from {username='slave'} at 127.0.0.1:47584
I20260812 19:06:04.160974 28557 raft_consensus.cc:482] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:04.161259 28557 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:04.166070 28557 raft_consensus.cc:504] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.167608 28557 leader_election.cc:288] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 3 election: Requested vote from peers 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:04.168501 28674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:04.168639 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80e6d885e328447ebd55ed5cf2690fe1" candidate_uuid: "4e11a4f6072b426693a6664f95af0907" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:04.168984 28674 raft_consensus.cc:3032] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:06:04.169225 28798 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:04.169247 28674 raft_consensus.cc:729] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:06:04.170177 28674 consensus_queue.cc:260] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.171452 28674 raft_consensus.cc:3037] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:04.176239 28798 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 3.
I20260812 19:06:04.189440 28498 leader_election.cc:302] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:04.192188 29000 raft_consensus.cc:2781] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:04.193272 29000 raft_consensus.cc:686] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Becoming Leader. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:04.195122 29000 consensus_queue.cc:237] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:04.203058 28674 raft_consensus.cc:2454] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e11a4f6072b426693a6664f95af0907 in term 3.
I20260812 19:06:04.208833 28392 catalog_manager.cc:5495] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 reported cstate change: term changed from 2 to 3, leader changed from 144c38de478a44859c4593c766e5b1da (127.26.205.66) to 4e11a4f6072b426693a6664f95af0907 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "4e11a4f6072b426693a6664f95af0907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:04.267380 28674 raft_consensus.cc:1261] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:04.268482 29000 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:04.280021 28798 raft_consensus.cc:1261] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:04.281178 29000 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:04.306139 28798 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:04.307192 29000 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:04.317714 28674 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:04.319005 29039 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:04.496951 28674 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:04.498117 29039 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:04.507573 28798 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:04.509357 29066 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:04.518940 28674 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:04.520252 29066 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:04.531236 28798 raft_consensus.cc:1261] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:04.532641 29063 consensus_queue.cc:1035] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:04.677189 28674 raft_consensus.cc:1261] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:04.678520 29063 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:04.686533 28798 raft_consensus.cc:1261] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Refusing update from remote peer 4e11a4f6072b426693a6664f95af0907: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:04.687885 29000 consensus_queue.cc:1035] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:06.854773 27445 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20260812 19:06:06.855417 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:06.862826 28557 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:06:06.863449 28557 raft_consensus.cc:595] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Received request to transfer leadership to TS 144c38de478a44859c4593c766e5b1da
I20260812 19:06:06.865861 28557 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:06:06.866350 28557 raft_consensus.cc:595] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Received request to transfer leadership to TS 0689ea7aff1049e28144361883e0ee37
I20260812 19:06:06.868498 28557 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:06:06.868945 28557 raft_consensus.cc:595] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Received request to transfer leadership to TS 0689ea7aff1049e28144361883e0ee37
I20260812 19:06:06.871129 28557 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "4e11a4f6072b426693a6664f95af0907"
mode: GRACEFUL
new_leader_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:47610
I20260812 19:06:06.871532 28557 raft_consensus.cc:595] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Received request to transfer leadership to TS 144c38de478a44859c4593c766e5b1da
I20260812 19:06:06.872416 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260812 19:06:06.872934 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:06.939468 29063 raft_consensus.cc:979] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:06:06.939924 29078 raft_consensus.cc:1067] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Signalling peer 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:06:06.941352 28795 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e41963dc3a804baebf22193068832e38"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:51890
I20260812 19:06:06.941816 28795 raft_consensus.cc:482] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:06.942126 28795 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:06:06.947214 28795 raft_consensus.cc:504] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:06.948997 28795 leader_election.cc:288] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 5 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:06:06.950132 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:06.950031 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e41963dc3a804baebf22193068832e38" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:06:06.950766 28675 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:06:06.950814 28557 raft_consensus.cc:3032] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:06:06.951273 28557 raft_consensus.cc:729] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:06.952116 28557 consensus_queue.cc:260] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:06.953466 28557 raft_consensus.cc:3037] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:06:06.957995 28675 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 5.
I20260812 19:06:06.959226 28738 leader_election.cc:302] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:06:06.960037 29037 raft_consensus.cc:2781] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:06:06.960583 28557 raft_consensus.cc:2454] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 5.
I20260812 19:06:06.961045 29037 raft_consensus.cc:686] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 5 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:06:06.961942 29037 consensus_queue.cc:237] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:06.968812 28391 catalog_manager.cc:5495] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 4 to 5, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 5 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:07.068516 29063 raft_consensus.cc:979] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:06:07.069034 29078 raft_consensus.cc:1067] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Signalling peer 0689ea7aff1049e28144361883e0ee37 to start an election
I20260812 19:06:07.070498 28795 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486"
dest_uuid: "0689ea7aff1049e28144361883e0ee37"
from {username='slave'} at 127.0.0.1:51890
I20260812 19:06:07.071000 28795 raft_consensus.cc:482] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:07.071259 28795 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.075968 28795 raft_consensus.cc:504] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.077406 28795 leader_election.cc:288] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381)
I20260812 19:06:07.078503 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:06:07.079063 28557 raft_consensus.cc:3032] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:06:07.078990 28675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbcb053c08004c6db171442ca1a3b486" candidate_uuid: "0689ea7aff1049e28144361883e0ee37" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "144c38de478a44859c4593c766e5b1da"
I20260812 19:06:07.079476 28557 raft_consensus.cc:729] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:07.079779 28675 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.080127 28557 consensus_queue.cc:260] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.081108 28557 raft_consensus.cc:3037] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.085510 28557 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:06:07.086460 28675 raft_consensus.cc:2454] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0689ea7aff1049e28144361883e0ee37 in term 3.
I20260812 19:06:07.086668 28735 leader_election.cc:302] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 4e11a4f6072b426693a6664f95af0907; no voters:
I20260812 19:06:07.087500 29037 raft_consensus.cc:2781] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:07.087881 29037 raft_consensus.cc:686] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Becoming Leader. State: Replica: 0689ea7aff1049e28144361883e0ee37, State: Running, Role: LEADER
I20260812 19:06:07.088480 29037 consensus_queue.cc:237] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.095501 28391 catalog_manager.cc:5495] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 reported cstate change: term changed from 2 to 3, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 0689ea7aff1049e28144361883e0ee37 (127.26.205.67). New cstate: current_term: 3 leader_uuid: "0689ea7aff1049e28144361883e0ee37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:07.124327 29063 raft_consensus.cc:979] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:06:07.124765 29063 raft_consensus.cc:1067] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Signalling peer 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:06:07.126159 28675 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:37466
I20260812 19:06:07.126653 28675 raft_consensus.cc:482] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:07.126915 28675 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.131565 28675 raft_consensus.cc:504] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.133324 28675 leader_election.cc:288] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 3 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:07.134164 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:06:07.134289 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfcd9c50dc4148f6be24136e2e58176c" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:07.134718 28557 raft_consensus.cc:3032] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:06:07.134732 28795 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.135025 28557 raft_consensus.cc:729] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:07.135659 28557 consensus_queue.cc:260] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.136595 28557 raft_consensus.cc:3037] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:07.139194 28795 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 3.
I20260812 19:06:07.140600 28620 leader_election.cc:302] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:06:07.141108 28557 raft_consensus.cc:2454] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 3.
I20260812 19:06:07.141347 29038 raft_consensus.cc:2781] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:07.142311 29038 raft_consensus.cc:686] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 3 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:06:07.143141 29038 consensus_queue.cc:237] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.149420 28391 catalog_manager.cc:5495] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 2 to 3, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 3 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:07.343405 29078 raft_consensus.cc:979] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: : Instructing follower 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:06:07.343851 29063 raft_consensus.cc:1067] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Signalling peer 144c38de478a44859c4593c766e5b1da to start an election
I20260812 19:06:07.345181 28675 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2"
dest_uuid: "144c38de478a44859c4593c766e5b1da"
from {username='slave'} at 127.0.0.1:37466
I20260812 19:06:07.345610 28675 raft_consensus.cc:482] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:07.345849 28675 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:07.350210 28675 raft_consensus.cc:504] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.351586 28675 leader_election.cc:288] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 4 election: Requested vote from peers 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075), 0689ea7aff1049e28144361883e0ee37 (127.26.205.67:37111)
I20260812 19:06:07.352388 28557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4e11a4f6072b426693a6664f95af0907"
I20260812 19:06:07.352519 28798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f830c1407d4a41b693949b889e16cea2" candidate_uuid: "144c38de478a44859c4593c766e5b1da" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0689ea7aff1049e28144361883e0ee37"
I20260812 19:06:07.352958 28557 raft_consensus.cc:3032] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:06:07.353003 28798 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:07.353359 28557 raft_consensus.cc:729] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e11a4f6072b426693a6664f95af0907, State: Running, Role: LEADER
I20260812 19:06:07.354043 28557 consensus_queue.cc:260] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.355204 28557 raft_consensus.cc:3037] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:06:07.359856 28798 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 4.
I20260812 19:06:07.360953 28620 leader_election.cc:302] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0689ea7aff1049e28144361883e0ee37, 144c38de478a44859c4593c766e5b1da; no voters:
I20260812 19:06:07.361285 28557 raft_consensus.cc:2454] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144c38de478a44859c4593c766e5b1da in term 4.
I20260812 19:06:07.361778 29038 raft_consensus.cc:2781] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:06:07.362277 29038 raft_consensus.cc:686] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 4 LEADER]: Becoming Leader. State: Replica: 144c38de478a44859c4593c766e5b1da, State: Running, Role: LEADER
I20260812 19:06:07.363034 29038 consensus_queue.cc:237] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } }
I20260812 19:06:07.370081 28675 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 5 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:06:07.369508 28391 catalog_manager.cc:5495] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da reported cstate change: term changed from 3 to 4, leader changed from 4e11a4f6072b426693a6664f95af0907 (127.26.205.65) to 144c38de478a44859c4593c766e5b1da (127.26.205.66). New cstate: current_term: 4 leader_uuid: "144c38de478a44859c4593c766e5b1da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:07.371181 29037 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:07.380656 28557 raft_consensus.cc:1261] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 5 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:06:07.382747 29079 consensus_queue.cc:1035] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:07.554697 28557 raft_consensus.cc:1261] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:07.555773 29038 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:07.564041 28798 raft_consensus.cc:1261] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:07.566016 29092 consensus_queue.cc:1035] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:07.648972 28557 raft_consensus.cc:1261] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:07.650192 29037 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:07.658802 28675 raft_consensus.cc:1261] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Refusing update from remote peer 0689ea7aff1049e28144361883e0ee37: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:06:07.660320 29099 consensus_queue.cc:1035] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "144c38de478a44859c4593c766e5b1da" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:07.752727 28798 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:07.754063 29092 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "0689ea7aff1049e28144361883e0ee37" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:07.762446 28557 raft_consensus.cc:1261] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Refusing update from remote peer 144c38de478a44859c4593c766e5b1da: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:06:07.764319 29105 consensus_queue.cc:1035] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e11a4f6072b426693a6664f95af0907" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:07.873811 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:07.880182 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:07.880657 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:08.881623 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:08.887036 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:08.887492 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:09.888327 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:09.893553 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:09.893987 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:10.894840 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:10.900215 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:10.900668 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:11.901604 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:11.907183 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:11.907661 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:12.908562 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:12.913820 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:12.914283 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:13.915182 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:13.920502 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:13.920975 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:14.922163 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:14.930132 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:14.930532 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:15.931375 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:15.937249 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260812 19:06:15.937677 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:16.967424 27445 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20260812 19:06:16.967926 27445 tablet_server.cc:178] TabletServer@127.26.205.65:0 shutting down...
I20260812 19:06:16.995458 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:06:16.996549 27445 tablet_replica.cc:330] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:16.997577 27445 raft_consensus.cc:2229] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:06:16.998543 27445 raft_consensus.cc:2258] T 12ccdb81cc92432ebc3e4359ac0ccdca P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.000854 27445 tablet_replica.cc:330] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.001610 27445 raft_consensus.cc:2229] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.002826 27445 raft_consensus.cc:2258] T de8d6d7dce784fc0add3406e0413b541 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.004647 27445 tablet_replica.cc:330] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.005129 27445 raft_consensus.cc:2229] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.005520 27445 raft_consensus.cc:2258] T dbcb053c08004c6db171442ca1a3b486 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.007333 27445 tablet_replica.cc:330] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.007815 27445 raft_consensus.cc:2229] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.008208 27445 raft_consensus.cc:2258] T e41963dc3a804baebf22193068832e38 P 4e11a4f6072b426693a6664f95af0907 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.009922 27445 tablet_replica.cc:330] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.010438 27445 raft_consensus.cc:2229] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.010839 27445 raft_consensus.cc:2258] T bfcd9c50dc4148f6be24136e2e58176c P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.012526 27445 tablet_replica.cc:330] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.012979 27445 raft_consensus.cc:2229] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.013376 27445 raft_consensus.cc:2258] T e8ae7ad9bcb945869efb079d7719a015 P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.015305 27445 tablet_replica.cc:330] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.015767 27445 raft_consensus.cc:2229] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.016191 27445 raft_consensus.cc:2258] T f830c1407d4a41b693949b889e16cea2 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.017904 27445 tablet_replica.cc:330] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.018409 27445 raft_consensus.cc:2229] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 4 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.019182 27445 raft_consensus.cc:2258] T 412149fd5f434de6a6f68f33580c7476 P 4e11a4f6072b426693a6664f95af0907 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.020843 27445 tablet_replica.cc:330] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.021282 27445 raft_consensus.cc:2229] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 3 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.022078 27445 raft_consensus.cc:2258] T 80e6d885e328447ebd55ed5cf2690fe1 P 4e11a4f6072b426693a6664f95af0907 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.023726 27445 tablet_replica.cc:330] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907: stopping tablet replica
I20260812 19:06:17.024127 27445 raft_consensus.cc:2229] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.024493 27445 raft_consensus.cc:2258] T a038b5a28e9c4240a43afb3dc50c742c P 4e11a4f6072b426693a6664f95af0907 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.056214 27445 tablet_server.cc:194] TabletServer@127.26.205.65:0 shutdown complete.
I20260812 19:06:17.076704 27445 tablet_server.cc:178] TabletServer@127.26.205.66:0 shutting down...
I20260812 19:06:17.103766 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:06:17.104478 27445 tablet_replica.cc:330] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.105118 27445 raft_consensus.cc:2229] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.105687 27445 raft_consensus.cc:2258] T dbcb053c08004c6db171442ca1a3b486 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.107604 27445 tablet_replica.cc:330] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.108171 27445 raft_consensus.cc:2229] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.108624 27445 raft_consensus.cc:2258] T 12ccdb81cc92432ebc3e4359ac0ccdca P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.110801 27445 tablet_replica.cc:330] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.111299 27445 raft_consensus.cc:2229] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.112002 27445 raft_consensus.cc:2258] T de8d6d7dce784fc0add3406e0413b541 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus is shut down!
W20260812 19:06:17.113144 28616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.205.65:44075: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 19:06:17.114173 27445 tablet_replica.cc:330] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.114842 27445 raft_consensus.cc:2229] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.115368 27445 raft_consensus.cc:2258] T e8ae7ad9bcb945869efb079d7719a015 P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus is shut down!
W20260812 19:06:17.116784 28616 consensus_peers.cc:487] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da -> Peer 4e11a4f6072b426693a6664f95af0907 (127.26.205.65:44075): Couldn't send request to peer 4e11a4f6072b426693a6664f95af0907. Status: Network error: Client connection negotiation failed: client connection to 127.26.205.65:44075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:17.117704 27445 tablet_replica.cc:330] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.118286 27445 raft_consensus.cc:2229] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 5 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.118804 27445 raft_consensus.cc:2258] T e41963dc3a804baebf22193068832e38 P 144c38de478a44859c4593c766e5b1da [term 5 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.120709 27445 tablet_replica.cc:330] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.121232 27445 raft_consensus.cc:2229] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 3 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.122071 27445 raft_consensus.cc:2258] T bfcd9c50dc4148f6be24136e2e58176c P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.124078 27445 tablet_replica.cc:330] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.124608 27445 raft_consensus.cc:2229] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 4 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.125407 27445 raft_consensus.cc:2258] T f830c1407d4a41b693949b889e16cea2 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.127205 27445 tablet_replica.cc:330] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.127668 27445 raft_consensus.cc:2229] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.128458 27445 raft_consensus.cc:2258] T a038b5a28e9c4240a43afb3dc50c742c P 144c38de478a44859c4593c766e5b1da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.130297 27445 tablet_replica.cc:330] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.130764 27445 raft_consensus.cc:2229] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.131191 27445 raft_consensus.cc:2258] T 80e6d885e328447ebd55ed5cf2690fe1 P 144c38de478a44859c4593c766e5b1da [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.133241 27445 tablet_replica.cc:330] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da: stopping tablet replica
I20260812 19:06:17.133677 27445 raft_consensus.cc:2229] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.134266 27445 raft_consensus.cc:2258] T 412149fd5f434de6a6f68f33580c7476 P 144c38de478a44859c4593c766e5b1da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.164474 27445 tablet_server.cc:194] TabletServer@127.26.205.66:0 shutdown complete.
I20260812 19:06:17.186671 27445 tablet_server.cc:178] TabletServer@127.26.205.67:0 shutting down...
W20260812 19:06:17.208482 28738 consensus_peers.cc:487] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 -> Peer 144c38de478a44859c4593c766e5b1da (127.26.205.66:41381): Couldn't send request to peer 144c38de478a44859c4593c766e5b1da. Status: Network error: Client connection negotiation failed: client connection to 127.26.205.66:41381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:17.216281 27445 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260812 19:06:17.217000 27445 tablet_replica.cc:330] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.217623 27445 raft_consensus.cc:2229] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.218137 27445 raft_consensus.cc:2258] T 12ccdb81cc92432ebc3e4359ac0ccdca P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.219934 27445 tablet_replica.cc:330] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.220448 27445 raft_consensus.cc:2229] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.220916 27445 raft_consensus.cc:2258] T f830c1407d4a41b693949b889e16cea2 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.222693 27445 tablet_replica.cc:330] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.223201 27445 raft_consensus.cc:2229] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 3 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.223989 27445 raft_consensus.cc:2258] T dbcb053c08004c6db171442ca1a3b486 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.225692 27445 tablet_replica.cc:330] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.226270 27445 raft_consensus.cc:2229] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.226698 27445 raft_consensus.cc:2258] T a038b5a28e9c4240a43afb3dc50c742c P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.228317 27445 tablet_replica.cc:330] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.228751 27445 raft_consensus.cc:2229] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.229226 27445 raft_consensus.cc:2258] T 412149fd5f434de6a6f68f33580c7476 P 0689ea7aff1049e28144361883e0ee37 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.231271 27445 tablet_replica.cc:330] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.231870 27445 raft_consensus.cc:2229] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.232290 27445 raft_consensus.cc:2258] T 80e6d885e328447ebd55ed5cf2690fe1 P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.234081 27445 tablet_replica.cc:330] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.234699 27445 raft_consensus.cc:2229] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.235208 27445 raft_consensus.cc:2258] T de8d6d7dce784fc0add3406e0413b541 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.236815 27445 tablet_replica.cc:330] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.237260 27445 raft_consensus.cc:2229] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 2 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.238095 27445 raft_consensus.cc:2258] T e8ae7ad9bcb945869efb079d7719a015 P 0689ea7aff1049e28144361883e0ee37 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.239761 27445 tablet_replica.cc:330] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.240204 27445 raft_consensus.cc:2229] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:17.240739 27445 raft_consensus.cc:2258] T bfcd9c50dc4148f6be24136e2e58176c P 0689ea7aff1049e28144361883e0ee37 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.242515 27445 tablet_replica.cc:330] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37: stopping tablet replica
I20260812 19:06:17.242954 27445 raft_consensus.cc:2229] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 5 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.243713 27445 raft_consensus.cc:2258] T e41963dc3a804baebf22193068832e38 P 0689ea7aff1049e28144361883e0ee37 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.273627 27445 tablet_server.cc:194] TabletServer@127.26.205.67:0 shutdown complete.
I20260812 19:06:17.293519 27445 master.cc:537] Master@127.26.205.126:34053 shutting down...
I20260812 19:06:17.316318 27445 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 1 LEADER]: Raft consensus shutting down.
I20260812 19:06:17.316879 27445 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:17.317209 27445 tablet_replica.cc:330] T 00000000000000000000000000000000 P c7d60abd20a8465c8cea95d10595273f: stopping tablet replica
I20260812 19:06:17.336557 27445 master.cc:552] Master@127.26.205.126:34053 shutdown complete.
0689ea7aff1049e28144361883e0ee37 1
144c38de478a44859c4593c766e5b1da 4
4e11a4f6072b426693a6664f95af0907 5
0689ea7aff1049e28144361883e0ee37 3
144c38de478a44859c4593c766e5b1da 4
4e11a4f6072b426693a6664f95af0907 3
0689ea7aff1049e28144361883e0ee37 1
144c38de478a44859c4593c766e5b1da 1
4e11a4f6072b426693a6664f95af0907 8
0689ea7aff1049e28144361883e0ee37 3
144c38de478a44859c4593c766e5b1da 3
4e11a4f6072b426693a6664f95af0907 4
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28637 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260812 19:06:17.380148 27445 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.26.205.126:35929
I20260812 19:06:17.381222 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:17.387214 29134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.387415 29136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.387948 29138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:17.388798 27445 server_base.cc:1034] running on GCE node
I20260812 19:06:17.389819 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:06:17.390033 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:06:17.390172 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561577390161 us; error 0 us; skew 500 ppm
I20260812 19:06:17.390698 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:06:17.398316 27445 webserver.cc:460] Webserver started at http://127.26.205.126:42941/ using document root <none> and password file <none>
I20260812 19:06:17.398846 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:17.399036 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:17.399322 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:17.400573 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/master-0-root/instance:
uuid: "f47470148eaf49acb3f5b4813eb35f3e"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.405365 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:06:17.408988 29145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.409785 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:06:17.410125 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/master-0-root
uuid: "f47470148eaf49acb3f5b4813eb35f3e"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.410467 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:17.423520 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:17.424796 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:17.462453 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.126:35929
I20260812 19:06:17.462581 29236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.126:35929 every 8 connection(s)
I20260812 19:06:17.466361 29239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:17.476460 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Bootstrap starting.
I20260812 19:06:17.480809 29239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:17.485466 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: No bootstrap required, opened a new log
I20260812 19:06:17.487615 29239 raft_consensus.cc:348] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER }
I20260812 19:06:17.488188 29239 raft_consensus.cc:374] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:17.488431 29239 raft_consensus.cc:729] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47470148eaf49acb3f5b4813eb35f3e, State: Initialized, Role: FOLLOWER
I20260812 19:06:17.488996 29239 consensus_queue.cc:260] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER }
I20260812 19:06:17.489432 29239 raft_consensus.cc:388] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 19:06:17.489661 29239 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 19:06:17.489928 29239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:17.494112 29239 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER }
I20260812 19:06:17.494618 29239 leader_election.cc:302] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f47470148eaf49acb3f5b4813eb35f3e; no voters:
I20260812 19:06:17.495723 29239 leader_election.cc:288] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 19:06:17.496107 29242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:17.497313 29242 raft_consensus.cc:686] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [term 1 LEADER]: Becoming Leader. State: Replica: f47470148eaf49acb3f5b4813eb35f3e, State: Running, Role: LEADER
I20260812 19:06:17.497999 29242 consensus_queue.cc:237] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER }
I20260812 19:06:17.498556 29239 sys_catalog.cc:564] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 19:06:17.502873 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47470148eaf49acb3f5b4813eb35f3e. Latest consensus state: current_term: 1 leader_uuid: "f47470148eaf49acb3f5b4813eb35f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER } }
I20260812 19:06:17.503027 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f47470148eaf49acb3f5b4813eb35f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47470148eaf49acb3f5b4813eb35f3e" member_type: VOTER } }
I20260812 19:06:17.503521 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:17.503667 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e [sys.catalog]: This master's current role is: LEADER
I20260812 19:06:17.506471 29254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 19:06:17.511659 29254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 19:06:17.514194 27445 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 19:06:17.520752 29254 catalog_manager.cc:1347] Generated new cluster ID: 95d881ca472949c3b64cf645a3ca0b9e
I20260812 19:06:17.521049 29254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 19:06:17.542183 29254 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 19:06:17.543874 29254 catalog_manager.cc:1504] Loading token signing keys...
I20260812 19:06:17.564925 29254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Generated new TSK 0
I20260812 19:06:17.565565 29254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 19:06:17.580797 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:17.587113 29268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.588105 29269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.590127 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:17.590404 27445 server_base.cc:1034] running on GCE node
I20260812 19:06:17.591368 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:06:17.591562 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:06:17.591732 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561577591714 us; error 0 us; skew 500 ppm
I20260812 19:06:17.592252 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:06:17.594712 27445 webserver.cc:460] Webserver started at http://127.26.205.65:35595/ using document root <none> and password file <none>
I20260812 19:06:17.595202 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:17.595391 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:17.595635 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:17.596699 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/instance:
uuid: "6dd313c6592d4f33bab314a73f28b007"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.601478 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 19:06:17.605105 29279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.605901 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 19:06:17.606243 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root
uuid: "6dd313c6592d4f33bab314a73f28b007"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.606539 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:17.622368 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:17.623605 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:17.625087 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:17.627415 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:17.627636 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.627856 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:17.628012 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.668884 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.65:44295
I20260812 19:06:17.668988 29379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.65:44295 every 8 connection(s)
I20260812 19:06:17.673789 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:17.681382 29387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.682775 29388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.688149 29390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:17.688840 27445 server_base.cc:1034] running on GCE node
I20260812 19:06:17.690142 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:06:17.690382 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:06:17.690528 29381 heartbeater.cc:347] Connected to a master server at 127.26.205.126:35929
I20260812 19:06:17.690573 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561577690556 us; error 0 us; skew 500 ppm
I20260812 19:06:17.691028 29381 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:17.691493 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:06:17.691901 29381 heartbeater.cc:511] Master 127.26.205.126:35929 requested a full tablet report, sending...
I20260812 19:06:17.694067 27445 webserver.cc:460] Webserver started at http://127.26.205.66:43209/ using document root <none> and password file <none>
I20260812 19:06:17.694252 29179 ts_manager.cc:194] Registered new tserver with Master: 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:17.694725 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:17.694977 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:17.695330 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:17.697192 29179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55498
I20260812 19:06:17.697188 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/instance:
uuid: "182784ca1eb74135b8f16d8894fa4b01"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.702894 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 19:06:17.706393 29396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.707266 27445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 19:06:17.707806 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root
uuid: "182784ca1eb74135b8f16d8894fa4b01"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.708240 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:17.720228 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:17.721427 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:17.722995 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:17.725278 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:17.725485 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.725696 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:17.725837 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.766345 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.66:43659
I20260812 19:06:17.766453 29496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.66:43659 every 8 connection(s)
I20260812 19:06:17.771116 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 19:06:17.778214 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.780202 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:17.782199 29515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:17.782352 27445 server_base.cc:1034] running on GCE node
I20260812 19:06:17.783757 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:06:17.784039 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:06:17.784247 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561577784230 us; error 0 us; skew 500 ppm
I20260812 19:06:17.784809 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:06:17.785147 29500 heartbeater.cc:347] Connected to a master server at 127.26.205.126:35929
I20260812 19:06:17.785485 29500 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:17.786302 29500 heartbeater.cc:511] Master 127.26.205.126:35929 requested a full tablet report, sending...
I20260812 19:06:17.787504 27445 webserver.cc:460] Webserver started at http://127.26.205.67:43259/ using document root <none> and password file <none>
I20260812 19:06:17.788026 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:17.788208 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:17.788448 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:17.788448 29179 ts_manager.cc:194] Registered new tserver with Master: 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:17.789812 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/instance:
uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.790202 29179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55514
I20260812 19:06:17.794479 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 19:06:17.797863 29525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.799784 27445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 19:06:17.800073 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root
uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
format_stamp: "Formatted at 2026-08-12 19:06:17 on dist-test-slave-vpvm"
I20260812 19:06:17.800341 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:17.818383 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:17.819609 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:17.821143 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:17.823571 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:17.823979 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.824249 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:17.824388 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:17.864080 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.67:44573
I20260812 19:06:17.864224 29622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.67:44573 every 8 connection(s)
I20260812 19:06:17.879166 29623 heartbeater.cc:347] Connected to a master server at 127.26.205.126:35929
I20260812 19:06:17.879555 29623 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:17.880353 29623 heartbeater.cc:511] Master 127.26.205.126:35929 requested a full tablet report, sending...
I20260812 19:06:17.882421 29179 ts_manager.cc:194] Registered new tserver with Master: dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:17.882643 27445 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015720705s
I20260812 19:06:17.884785 29179 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55520
I20260812 19:06:17.885542 27445 test_util.cc:268] Using random seed: 1211637583
I20260812 19:06:17.914458 29179 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55532:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 19:06:17.921108 29179 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 19:06:18.133041 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc899036ba4e0d85662880b5e35132 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260812 19:06:18.134641 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc899036ba4e0d85662880b5e35132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.135118 29561 tablet_service.cc:1467] Processing CreateTablet for tablet e87477250ac44a318f7c0aef2d89fa36 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260812 19:06:18.136485 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87477250ac44a318f7c0aef2d89fa36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.138962 29444 tablet_service.cc:1467] Processing CreateTablet for tablet fd63b9965cdb4b568fd3b2860f8c2e61 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260812 19:06:18.139128 29559 tablet_service.cc:1467] Processing CreateTablet for tablet c89f8a2f65164cd08d1e760f9005e496 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260812 19:06:18.140424 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd63b9965cdb4b568fd3b2860f8c2e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.140496 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89f8a2f65164cd08d1e760f9005e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.141386 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 6469507c6875443ebd9d4c40bb47a54a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260812 19:06:18.142776 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6469507c6875443ebd9d4c40bb47a54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.143170 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 37fc905b65e94f9c9ad9b1bdc4f62986 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260812 19:06:18.144501 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fc905b65e94f9c9ad9b1bdc4f62986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.144351 29442 tablet_service.cc:1467] Processing CreateTablet for tablet e87477250ac44a318f7c0aef2d89fa36 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260812 19:06:18.145678 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87477250ac44a318f7c0aef2d89fa36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.147085 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 6469507c6875443ebd9d4c40bb47a54a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260812 19:06:18.148358 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6469507c6875443ebd9d4c40bb47a54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.151140 29433 tablet_service.cc:1467] Processing CreateTablet for tablet a5c317250947485cab086b4bfab851bd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260812 19:06:18.152014 29566 tablet_service.cc:1467] Processing CreateTablet for tablet fd63b9965cdb4b568fd3b2860f8c2e61 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260812 19:06:18.152539 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c317250947485cab086b4bfab851bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.152273 29437 tablet_service.cc:1467] Processing CreateTablet for tablet c89f8a2f65164cd08d1e760f9005e496 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260812 19:06:18.153573 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89f8a2f65164cd08d1e760f9005e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.151155 29438 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd0e82dc2545e7867d77ada5cfb445 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260812 19:06:18.155396 29438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd0e82dc2545e7867d77ada5cfb445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.155031 29436 tablet_service.cc:1467] Processing CreateTablet for tablet 37fc905b65e94f9c9ad9b1bdc4f62986 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260812 19:06:18.143173 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc899036ba4e0d85662880b5e35132 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260812 19:06:18.158865 29319 tablet_service.cc:1467] Processing CreateTablet for tablet e87477250ac44a318f7c0aef2d89fa36 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260812 19:06:18.159212 29324 tablet_service.cc:1467] Processing CreateTablet for tablet fd63b9965cdb4b568fd3b2860f8c2e61 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260812 19:06:18.160238 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87477250ac44a318f7c0aef2d89fa36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.160557 29430 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3af04408974eeab695361c3b2a8410 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260812 19:06:18.161258 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 97cc899036ba4e0d85662880b5e35132 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260812 19:06:18.162569 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc899036ba4e0d85662880b5e35132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.158778 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fc905b65e94f9c9ad9b1bdc4f62986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.166347 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3af04408974eeab695361c3b2a8410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.167150 29316 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd0e82dc2545e7867d77ada5cfb445 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260812 19:06:18.153398 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 79bd0e82dc2545e7867d77ada5cfb445 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260812 19:06:18.168474 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd0e82dc2545e7867d77ada5cfb445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.168758 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd0e82dc2545e7867d77ada5cfb445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.170313 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc899036ba4e0d85662880b5e35132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.171160 29315 tablet_service.cc:1467] Processing CreateTablet for tablet c89f8a2f65164cd08d1e760f9005e496 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260812 19:06:18.172456 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89f8a2f65164cd08d1e760f9005e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.174296 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd63b9965cdb4b568fd3b2860f8c2e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.175139 29314 tablet_service.cc:1467] Processing CreateTablet for tablet 37fc905b65e94f9c9ad9b1bdc4f62986 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260812 19:06:18.176800 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37fc905b65e94f9c9ad9b1bdc4f62986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.178090 29312 tablet_service.cc:1467] Processing CreateTablet for tablet a5c317250947485cab086b4bfab851bd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260812 19:06:18.179397 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c317250947485cab086b4bfab851bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.181859 29556 tablet_service.cc:1467] Processing CreateTablet for tablet a5c317250947485cab086b4bfab851bd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260812 19:06:18.183262 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c317250947485cab086b4bfab851bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.184865 29313 tablet_service.cc:1467] Processing CreateTablet for tablet 6469507c6875443ebd9d4c40bb47a54a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260812 19:06:18.186179 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6469507c6875443ebd9d4c40bb47a54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.183058 29310 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3af04408974eeab695361c3b2a8410 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260812 19:06:18.194180 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3af04408974eeab695361c3b2a8410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.194832 29554 tablet_service.cc:1467] Processing CreateTablet for tablet 4d3af04408974eeab695361c3b2a8410 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260812 19:06:18.198227 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd63b9965cdb4b568fd3b2860f8c2e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.198238 29428 tablet_service.cc:1467] Processing CreateTablet for tablet f6dae5f1cee3498787a4a28802d59d16 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260812 19:06:18.200009 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dae5f1cee3498787a4a28802d59d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.202905 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3af04408974eeab695361c3b2a8410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.210199 29553 tablet_service.cc:1467] Processing CreateTablet for tablet f6dae5f1cee3498787a4a28802d59d16 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260812 19:06:18.211627 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dae5f1cee3498787a4a28802d59d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.222908 29307 tablet_service.cc:1467] Processing CreateTablet for tablet f6dae5f1cee3498787a4a28802d59d16 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260812 19:06:18.225019 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dae5f1cee3498787a4a28802d59d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.263653 29649 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:18.280822 29652 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:18.281440 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 1226d05dc64043078e42cf470348655d (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260812 19:06:18.282925 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1226d05dc64043078e42cf470348655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.291658 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 75e1d0973f2d43b6affb3b455c7789a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260812 19:06:18.296115 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7629c8adbd447596c2e821c5ac0a99 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260812 19:06:18.305053 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7629c8adbd447596c2e821c5ac0a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.306169 29430 tablet_service.cc:1467] Processing CreateTablet for tablet c197fd0960e5419fafd6f7eec6f8959a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260812 19:06:18.307706 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c197fd0960e5419fafd6f7eec6f8959a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.307601 29444 tablet_service.cc:1467] Processing CreateTablet for tablet 7adcd8cac12a4350a441ca5a35090ec9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260812 19:06:18.309234 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adcd8cac12a4350a441ca5a35090ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.312740 29560 tablet_service.cc:1467] Processing CreateTablet for tablet c197fd0960e5419fafd6f7eec6f8959a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260812 19:06:18.318249 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e1d0973f2d43b6affb3b455c7789a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.322436 29553 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7629c8adbd447596c2e821c5ac0a99 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260812 19:06:18.323875 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7629c8adbd447596c2e821c5ac0a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.332844 29566 tablet_service.cc:1467] Processing CreateTablet for tablet 7adcd8cac12a4350a441ca5a35090ec9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260812 19:06:18.297094 29324 tablet_service.cc:1467] Processing CreateTablet for tablet 8b7629c8adbd447596c2e821c5ac0a99 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260812 19:06:18.337348 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7629c8adbd447596c2e821c5ac0a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.351361 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8ba27ebf754ac098699f3efab22620 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260812 19:06:18.335620 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 1226d05dc64043078e42cf470348655d (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260812 19:06:18.339895 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 75e1d0973f2d43b6affb3b455c7789a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260812 19:06:18.334875 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c197fd0960e5419fafd6f7eec6f8959a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.342406 29320 tablet_service.cc:1467] Processing CreateTablet for tablet c197fd0960e5419fafd6f7eec6f8959a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260812 19:06:18.337404 29650 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:18.353261 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adcd8cac12a4350a441ca5a35090ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.355093 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8ba27ebf754ac098699f3efab22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.346865 29316 tablet_service.cc:1467] Processing CreateTablet for tablet 1226d05dc64043078e42cf470348655d (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260812 19:06:18.315024 29652 tablet_bootstrap.cc:654] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.350838 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 47762c70d86d4362baf4ba75331efe8a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260812 19:06:18.357582 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47762c70d86d4362baf4ba75331efe8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.361475 29436 tablet_service.cc:1467] Processing CreateTablet for tablet 75e1d0973f2d43b6affb3b455c7789a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260812 19:06:18.372578 29444 tablet_service.cc:1467] Processing CreateTablet for tablet 47762c70d86d4362baf4ba75331efe8a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260812 19:06:18.380767 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 5616609c9dad428f93e5627a7ca84f2a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260812 19:06:18.380767 29430 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8ba27ebf754ac098699f3efab22620 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260812 19:06:18.384227 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5616609c9dad428f93e5627a7ca84f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.384456 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 5616609c9dad428f93e5627a7ca84f2a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260812 19:06:18.384891 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47762c70d86d4362baf4ba75331efe8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.385547 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1226d05dc64043078e42cf470348655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.385694 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5616609c9dad428f93e5627a7ca84f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.390403 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8ba27ebf754ac098699f3efab22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.392804 29650 tablet_bootstrap.cc:654] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.399657 29433 tablet_service.cc:1467] Processing CreateTablet for tablet dfb5c8ad869c44379e8aa81d22549ae5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260812 19:06:18.410157 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 7adcd8cac12a4350a441ca5a35090ec9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260812 19:06:18.414723 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260812 19:06:18.419266 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 647dfc8bad934787a924a06988dda474 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260812 19:06:18.413177 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 647dfc8bad934787a924a06988dda474 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260812 19:06:18.426359 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e1d0973f2d43b6affb3b455c7789a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.416875 29560 tablet_service.cc:1467] Processing CreateTablet for tablet dfb5c8ad869c44379e8aa81d22549ae5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260812 19:06:18.427282 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647dfc8bad934787a924a06988dda474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.427865 29312 tablet_service.cc:1467] Processing CreateTablet for tablet 47762c70d86d4362baf4ba75331efe8a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260812 19:06:18.429025 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75e1d0973f2d43b6affb3b455c7789a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.430090 29314 tablet_service.cc:1467] Processing CreateTablet for tablet 6d8ba27ebf754ac098699f3efab22620 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260812 19:06:18.435935 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8ba27ebf754ac098699f3efab22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.436558 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47762c70d86d4362baf4ba75331efe8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.432292 29313 tablet_service.cc:1467] Processing CreateTablet for tablet 5616609c9dad428f93e5627a7ca84f2a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260812 19:06:18.405292 29649 tablet_bootstrap.cc:654] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.437891 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5616609c9dad428f93e5627a7ca84f2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.437587 29442 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed56ce03a49908ad6f468e43b5402 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260812 19:06:18.442243 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7adcd8cac12a4350a441ca5a35090ec9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.444878 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c197fd0960e5419fafd6f7eec6f8959a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.448275 29438 tablet_service.cc:1467] Processing CreateTablet for tablet 69bfa848552945a18495c13ca6707d65 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260812 19:06:18.454147 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1226d05dc64043078e42cf470348655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.455137 29566 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed56ce03a49908ad6f468e43b5402 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260812 19:06:18.456431 29310 tablet_service.cc:1467] Processing CreateTablet for tablet dfb5c8ad869c44379e8aa81d22549ae5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260812 19:06:18.458256 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb5c8ad869c44379e8aa81d22549ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.458954 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb5c8ad869c44379e8aa81d22549ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.487537 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d28ed56ce03a49908ad6f468e43b5402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.488189 29652 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:18.488686 29652 ts_tablet_manager.cc:1397] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.208s user 0.011s sys 0.004s
I20260812 19:06:18.488809 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9f6c46c1a5494aa914b41b3a1616a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.491611 29652 raft_consensus.cc:348] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.493076 29652 raft_consensus.cc:374] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.493542 29652 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.494580 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb5c8ad869c44379e8aa81d22549ae5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.494494 29652 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.499138 29324 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260812 19:06:18.500520 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9f6c46c1a5494aa914b41b3a1616a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 19:06:18.510067 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:18.469385 29554 tablet_service.cc:1467] Processing CreateTablet for tablet 69bfa848552945a18495c13ca6707d65 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260812 19:06:18.523480 29649 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:18.524050 29649 ts_tablet_manager.cc:1397] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.261s user 0.023s sys 0.023s
I20260812 19:06:18.526780 29649 raft_consensus.cc:348] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.473609 29319 tablet_service.cc:1467] Processing CreateTablet for tablet 647dfc8bad934787a924a06988dda474 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260812 19:06:18.527558 29649 raft_consensus.cc:374] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.527956 29649 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.528680 29649 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.530467 29561 tablet_service.cc:1467] Processing CreateTablet for tablet bf312d0cb2a6414794d20e98e1edf8ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260812 19:06:18.531132 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 59710c65553f42188b78bc9bc0324542 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260812 19:06:18.534199 29316 tablet_service.cc:1467] Processing CreateTablet for tablet 69bfa848552945a18495c13ca6707d65 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260812 19:06:18.534421 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 6e90268109844f62bf1e1ee425b2946a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260812 19:06:18.536747 29428 tablet_service.cc:1467] Processing CreateTablet for tablet bf312d0cb2a6414794d20e98e1edf8ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260812 19:06:18.537731 29315 tablet_service.cc:1467] Processing CreateTablet for tablet bf312d0cb2a6414794d20e98e1edf8ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260812 19:06:18.539180 29430 tablet_service.cc:1467] Processing CreateTablet for tablet 6e90268109844f62bf1e1ee425b2946a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260812 19:06:18.546991 29553 tablet_service.cc:1467] Processing CreateTablet for tablet 59710c65553f42188b78bc9bc0324542 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260812 19:06:18.552627 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 6e90268109844f62bf1e1ee425b2946a (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260812 19:06:18.538815 29313 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed56ce03a49908ad6f468e43b5402 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260812 19:06:18.559836 29381 heartbeater.cc:503] Master 127.26.205.126:35929 was elected leader, sending a full tablet report...
I20260812 19:06:18.559958 29652 ts_tablet_manager.cc:1428] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.071s user 0.003s sys 0.012s
I20260812 19:06:18.581820 29652 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:18.570170 29438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bfa848552945a18495c13ca6707d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.583163 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf312d0cb2a6414794d20e98e1edf8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.542071 29559 tablet_service.cc:1467] Processing CreateTablet for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260812 19:06:18.586237 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf312d0cb2a6414794d20e98e1edf8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.586431 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59710c65553f42188b78bc9bc0324542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.556886 29558 tablet_service.cc:1467] Processing CreateTablet for tablet b83150d7d32c400083ab43de4f67f607 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260812 19:06:18.572769 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647dfc8bad934787a924a06988dda474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.572328 29436 tablet_service.cc:1467] Processing CreateTablet for tablet b83150d7d32c400083ab43de4f67f607 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260812 19:06:18.601374 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bfa848552945a18495c13ca6707d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.594033 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 62647baf8dbe49b9b57e1f296bfa85e6 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260812 19:06:18.604977 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e90268109844f62bf1e1ee425b2946a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.604271 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59710c65553f42188b78bc9bc0324542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.610814 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9f6c46c1a5494aa914b41b3a1616a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.611450 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e90268109844f62bf1e1ee425b2946a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.612476 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d28ed56ce03a49908ad6f468e43b5402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.614722 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bfa848552945a18495c13ca6707d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.615669 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 62647baf8dbe49b9b57e1f296bfa85e6 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260812 19:06:18.617143 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83150d7d32c400083ab43de4f67f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.622074 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647dfc8bad934787a924a06988dda474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.622797 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d28ed56ce03a49908ad6f468e43b5402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.626456 29623 heartbeater.cc:503] Master 127.26.205.126:35929 was elected leader, sending a full tablet report...
I20260812 19:06:18.626380 29312 tablet_service.cc:1467] Processing CreateTablet for tablet 015555682f5f4cf293ca3718be163873 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260812 19:06:18.627699 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015555682f5f4cf293ca3718be163873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.629667 29652 tablet_bootstrap.cc:654] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.613718 29314 tablet_service.cc:1467] Processing CreateTablet for tablet b83150d7d32c400083ab43de4f67f607 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260812 19:06:18.634446 29649 ts_tablet_manager.cc:1428] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.110s user 0.018s sys 0.037s
I20260812 19:06:18.636270 29649 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:18.646174 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62647baf8dbe49b9b57e1f296bfa85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.645987 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf312d0cb2a6414794d20e98e1edf8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.647629 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83150d7d32c400083ab43de4f67f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.654670 29650 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:18.666110 29444 tablet_service.cc:1467] Processing CreateTablet for tablet 015555682f5f4cf293ca3718be163873 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260812 19:06:18.650132 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 62647baf8dbe49b9b57e1f296bfa85e6 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260812 19:06:18.688793 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 015555682f5f4cf293ca3718be163873 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260812 19:06:18.684449 29433 tablet_service.cc:1467] Processing CreateTablet for tablet c0dc7d63c9244e56b10c73fbe2608457 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260812 19:06:18.690757 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e90268109844f62bf1e1ee425b2946a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.699872 29562 tablet_service.cc:1467] Processing CreateTablet for tablet c0dc7d63c9244e56b10c73fbe2608457 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260812 19:06:18.700428 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b83150d7d32c400083ab43de4f67f607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.700681 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015555682f5f4cf293ca3718be163873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.699481 29319 tablet_service.cc:1467] Processing CreateTablet for tablet c0dc7d63c9244e56b10c73fbe2608457 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260812 19:06:18.703299 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0dc7d63c9244e56b10c73fbe2608457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.706300 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62647baf8dbe49b9b57e1f296bfa85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.709216 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0dc7d63c9244e56b10c73fbe2608457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.710453 29650 ts_tablet_manager.cc:1397] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.373s user 0.015s sys 0.003s
I20260812 19:06:18.709870 29561 tablet_service.cc:1467] Processing CreateTablet for tablet da12498df45141cd80e5cb36e68775c5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260812 19:06:18.722309 29310 tablet_service.cc:1467] Processing CreateTablet for tablet 59710c65553f42188b78bc9bc0324542 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260812 19:06:18.723779 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59710c65553f42188b78bc9bc0324542. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.724153 29437 tablet_service.cc:1467] Processing CreateTablet for tablet da12498df45141cd80e5cb36e68775c5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260812 19:06:18.725384 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da12498df45141cd80e5cb36e68775c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.690635 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62647baf8dbe49b9b57e1f296bfa85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.734553 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0dc7d63c9244e56b10c73fbe2608457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.735188 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da12498df45141cd80e5cb36e68775c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.769362 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015555682f5f4cf293ca3718be163873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.773140 29316 tablet_service.cc:1467] Processing CreateTablet for tablet da12498df45141cd80e5cb36e68775c5 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260812 19:06:18.774528 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da12498df45141cd80e5cb36e68775c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.817443 29657 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:18.826262 29500 heartbeater.cc:503] Master 127.26.205.126:35929 was elected leader, sending a full tablet report...
I20260812 19:06:18.828609 29657 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.830911 29652 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:18.831576 29652 ts_tablet_manager.cc:1397] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.250s user 0.009s sys 0.022s
I20260812 19:06:18.844344 29649 tablet_bootstrap.cc:654] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.844875 29652 raft_consensus.cc:348] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:18.850616 29314 tablet_service.cc:1467] Processing CreateTablet for tablet d0dfbce6d2e3431ebc71db36ae455105 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260812 19:06:18.852005 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0dfbce6d2e3431ebc71db36ae455105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.864712 29657 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:18.867053 29442 tablet_service.cc:1467] Processing CreateTablet for tablet d0dfbce6d2e3431ebc71db36ae455105 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260812 19:06:18.868574 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0dfbce6d2e3431ebc71db36ae455105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.845816 29652 raft_consensus.cc:374] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.870306 29652 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.870378 29650 raft_consensus.cc:348] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.871385 29650 raft_consensus.cc:374] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.871721 29650 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.872573 29650 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.879626 29313 tablet_service.cc:1467] Processing CreateTablet for tablet 4737efcfbbb9484781f7335f47b8c2ae (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260812 19:06:18.883713 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 005defc807dc47d881add64226efb7ce (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260812 19:06:18.884325 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 69508446a398466a943e7ad290aa8e0c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260812 19:06:18.886477 29652 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:18.927357 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4737efcfbbb9484781f7335f47b8c2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.928987 29652 ts_tablet_manager.cc:1428] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.097s user 0.006s sys 0.008s
I20260812 19:06:18.929900 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69508446a398466a943e7ad290aa8e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.895418 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 69508446a398466a943e7ad290aa8e0c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260812 19:06:18.902622 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 69508446a398466a943e7ad290aa8e0c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260812 19:06:18.911463 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 005defc807dc47d881add64226efb7ce (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260812 19:06:18.913089 29433 tablet_service.cc:1467] Processing CreateTablet for tablet 4737efcfbbb9484781f7335f47b8c2ae (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260812 19:06:18.914000 29559 tablet_service.cc:1467] Processing CreateTablet for tablet 4737efcfbbb9484781f7335f47b8c2ae (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260812 19:06:18.916059 29319 tablet_service.cc:1467] Processing CreateTablet for tablet d05c973d4068462fa79a7bec57fc6e89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260812 19:06:18.919420 29428 tablet_service.cc:1467] Processing CreateTablet for tablet d05c973d4068462fa79a7bec57fc6e89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260812 19:06:18.937717 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4737efcfbbb9484781f7335f47b8c2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.938227 29652 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:18.939834 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05c973d4068462fa79a7bec57fc6e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.940768 29649 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:18.941219 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005defc807dc47d881add64226efb7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.941247 29649 ts_tablet_manager.cc:1397] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.305s user 0.018s sys 0.008s
I20260812 19:06:18.946312 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69508446a398466a943e7ad290aa8e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.946535 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69508446a398466a943e7ad290aa8e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.949795 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4737efcfbbb9484781f7335f47b8c2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.950804 29553 tablet_service.cc:1467] Processing CreateTablet for tablet db322776a340462b91983d8e448cfe39 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260812 19:06:18.952023 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db322776a340462b91983d8e448cfe39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.960000 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 637fa5c7fa824870a0da61e2bc8400af (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260812 19:06:18.946460 29649 raft_consensus.cc:348] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.961242 29649 raft_consensus.cc:374] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:18.961462 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637fa5c7fa824870a0da61e2bc8400af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.963261 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005defc807dc47d881add64226efb7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.963693 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 637fa5c7fa824870a0da61e2bc8400af (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260812 19:06:18.964154 29652 tablet_bootstrap.cc:654] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:18.964406 29649 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:18.964915 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637fa5c7fa824870a0da61e2bc8400af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.965417 29649 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.970314 29566 tablet_service.cc:1467] Processing CreateTablet for tablet d0dfbce6d2e3431ebc71db36ae455105 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260812 19:06:18.970877 29430 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebf3eea25534182aa1c9f148898dc8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260812 19:06:18.971577 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0dfbce6d2e3431ebc71db36ae455105. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.971442 29313 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebf3eea25534182aa1c9f148898dc8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260812 19:06:18.972126 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebf3eea25534182aa1c9f148898dc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.972393 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:18.974138 29282 leader_election.cc:341] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:18.920460 29438 tablet_service.cc:1467] Processing CreateTablet for tablet db322776a340462b91983d8e448cfe39 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260812 19:06:18.921491 29557 tablet_service.cc:1467] Processing CreateTablet for tablet 005defc807dc47d881add64226efb7ce (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260812 19:06:18.981195 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005defc807dc47d881add64226efb7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.924160 29444 tablet_service.cc:1467] Processing CreateTablet for tablet 637fa5c7fa824870a0da61e2bc8400af (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260812 19:06:18.982829 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637fa5c7fa824870a0da61e2bc8400af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:18.925900 29324 tablet_service.cc:1467] Processing CreateTablet for tablet db322776a340462b91983d8e448cfe39 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260812 19:06:18.985316 29658 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:18.985791 29658 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:18.993762 29658 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:18.999123 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db322776a340462b91983d8e448cfe39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.026513 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 8987f70c28f749d8b5f5ef83f3349efd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260812 19:06:19.032663 29650 ts_tablet_manager.cc:1428] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.322s user 0.009s sys 0.022s
I20260812 19:06:19.019281 29438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db322776a340462b91983d8e448cfe39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.051177 29649 ts_tablet_manager.cc:1428] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.107s user 0.007s sys 0.000s
I20260812 19:06:19.055393 29650 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:19.040506 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebf3eea25534182aa1c9f148898dc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.066973 29649 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.070206 29316 tablet_service.cc:1467] Processing CreateTablet for tablet 8987f70c28f749d8b5f5ef83f3349efd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260812 19:06:19.044277 29561 tablet_service.cc:1467] Processing CreateTablet for tablet d05c973d4068462fa79a7bec57fc6e89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260812 19:06:19.077425 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05c973d4068462fa79a7bec57fc6e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.079512 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 8987f70c28f749d8b5f5ef83f3349efd (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260812 19:06:19.080791 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8987f70c28f749d8b5f5ef83f3349efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.074224 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8987f70c28f749d8b5f5ef83f3349efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.084810 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:19.085850 29672 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.086256 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.086354 29672 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:19.088209 29281 leader_election.cc:341] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.088652 29281 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.090701 29657 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.095044 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebf3eea25534182aa1c9f148898dc8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260812 19:06:19.096329 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebf3eea25534182aa1c9f148898dc8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 19:06:19.096946 29528 leader_election.cc:341] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.098539 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260812 19:06:19.099792 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05c973d4068462fa79a7bec57fc6e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.106585 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260812 19:06:19.107688 29649 tablet_bootstrap.cc:654] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.107868 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.107667 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 14e182b70d344161a275142a344cd1ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260812 19:06:19.111280 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.116367 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e182b70d344161a275142a344cd1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.099680 29672 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.122426 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8987f70c28f749d8b5f5ef83f3349efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.122314 29312 tablet_service.cc:1467] Processing CreateTablet for tablet ffa1accbf0a749988850cc0f47a8efb0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260812 19:06:19.123513 29652 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:19.123663 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa1accbf0a749988850cc0f47a8efb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.141661 29652 ts_tablet_manager.cc:1397] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.203s user 0.018s sys 0.011s
I20260812 19:06:19.147174 29314 tablet_service.cc:1467] Processing CreateTablet for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260812 19:06:19.148500 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.147799 29650 tablet_bootstrap.cc:654] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.178143 29433 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbee78e8845445eb3a4cbba1399e988 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260812 19:06:19.147927 29310 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbee78e8845445eb3a4cbba1399e988 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260812 19:06:19.179085 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbee78e8845445eb3a4cbba1399e988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.183073 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbee78e8845445eb3a4cbba1399e988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.160564 29561 tablet_service.cc:1467] Processing CreateTablet for tablet 14e182b70d344161a275142a344cd1ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260812 19:06:19.163614 29559 tablet_service.cc:1467] Processing CreateTablet for tablet ffa1accbf0a749988850cc0f47a8efb0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260812 19:06:19.166661 29652 raft_consensus.cc:348] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.188405 29652 raft_consensus.cc:374] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.168665 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc924748e9a482e8536297e097a5226 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260812 19:06:19.168998 29566 tablet_service.cc:1467] Processing CreateTablet for tablet 8cbee78e8845445eb3a4cbba1399e988 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260812 19:06:19.171227 29436 tablet_service.cc:1467] Processing CreateTablet for tablet ffa1accbf0a749988850cc0f47a8efb0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260812 19:06:19.188098 29554 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc924748e9a482e8536297e097a5226 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260812 19:06:19.167848 29435 tablet_service.cc:1467] Processing CreateTablet for tablet 14e182b70d344161a275142a344cd1ba (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260812 19:06:19.188774 29652 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.191088 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc924748e9a482e8536297e097a5226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.188011 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa1accbf0a749988850cc0f47a8efb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.180574 29649 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.197408 29652 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.202302 29657 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.203899 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 0adb15c8e8a8443a9c36474d5714f204 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260812 19:06:19.206465 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa1accbf0a749988850cc0f47a8efb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.208015 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e182b70d344161a275142a344cd1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.213639 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 2743b6275ccc4fbdbe5f71c667adee8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260812 19:06:19.214449 29553 tablet_service.cc:1467] Processing CreateTablet for tablet ed9c55b662464bbd8e692e47a2f7cc2c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260812 19:06:19.219012 29437 tablet_service.cc:1467] Processing CreateTablet for tablet 0adb15c8e8a8443a9c36474d5714f204 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260812 19:06:19.221199 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 514939f267f2450faf798117d94726ab (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260812 19:06:19.227658 29657 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.217828 29557 tablet_service.cc:1467] Processing CreateTablet for tablet fe3191acdfe44a019be79eda30b9e79b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260812 19:06:19.246547 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0adb15c8e8a8443a9c36474d5714f204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.246650 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9c55b662464bbd8e692e47a2f7cc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.266530 29324 tablet_service.cc:1467] Processing CreateTablet for tablet 2743b6275ccc4fbdbe5f71c667adee8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260812 19:06:19.267094 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbee78e8845445eb3a4cbba1399e988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.247218 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e182b70d344161a275142a344cd1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.267956 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2743b6275ccc4fbdbe5f71c667adee8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.271118 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:19.272541 29282 leader_election.cc:341] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.266525 29562 tablet_service.cc:1467] Processing CreateTablet for tablet 0adb15c8e8a8443a9c36474d5714f204 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260812 19:06:19.254114 29428 tablet_service.cc:1467] Processing CreateTablet for tablet 2743b6275ccc4fbdbe5f71c667adee8b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260812 19:06:19.247982 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.290511 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0adb15c8e8a8443a9c36474d5714f204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.194480 29430 tablet_service.cc:1467] Processing CreateTablet for tablet 3cc924748e9a482e8536297e097a5226 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260812 19:06:19.291141 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.291725 29430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc924748e9a482e8536297e097a5226. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 19:06:19.292632 29530 leader_election.cc:341] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.293102 29530 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:19.294127 29658 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:19.295439 29403 leader_election.cc:341] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.297240 29319 tablet_service.cc:1467] Processing CreateTablet for tablet ed9c55b662464bbd8e692e47a2f7cc2c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260812 19:06:19.298535 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9c55b662464bbd8e692e47a2f7cc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.298632 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2743b6275ccc4fbdbe5f71c667adee8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.260303 29442 tablet_service.cc:1467] Processing CreateTablet for tablet ed9c55b662464bbd8e692e47a2f7cc2c (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260812 19:06:19.229259 29649 ts_tablet_manager.cc:1397] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.163s user 0.014s sys 0.002s
I20260812 19:06:19.265264 29657 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:19.301132 29657 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.301573 29657 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.303588 29657 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.304337 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:19.306210 29649 raft_consensus.cc:348] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.307010 29649 raft_consensus.cc:374] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.307391 29649 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.307324 29658 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.308235 29658 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.308439 29649 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.313408 29658 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:19.314738 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.315364 29580 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
W20260812 19:06:19.316238 29282 leader_election.cc:341] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.317155 29399 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:19.317828 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.318274 29672 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:19.318468 29341 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:19.319300 29672 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.319711 29672 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.319618 29530 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:19.320288 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:19.320492 29658 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:19.321260 29658 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.321601 29658 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.321699 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2743b6275ccc4fbdbe5f71c667adee8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 19:06:19.326273 29528 leader_election.cc:341] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.326921 29672 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.319482 29316 tablet_service.cc:1467] Processing CreateTablet for tablet fe3191acdfe44a019be79eda30b9e79b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260812 19:06:19.334841 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9c55b662464bbd8e692e47a2f7cc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.336408 29558 tablet_service.cc:1467] Processing CreateTablet for tablet f55b37aa5102423ebbccef3f872d85e8 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260812 19:06:19.337112 29672 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.338927 29652 ts_tablet_manager.cc:1428] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.188s user 0.005s sys 0.010s
I20260812 19:06:19.339759 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 514939f267f2450faf798117d94726ab (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260812 19:06:19.341935 29320 tablet_service.cc:1467] Processing CreateTablet for tablet 514939f267f2450faf798117d94726ab (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260812 19:06:19.342832 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:19.344236 29403 leader_election.cc:341] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.344981 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:19.345611 29580 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.345242 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe3191acdfe44a019be79eda30b9e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.341429 29433 tablet_service.cc:1467] Processing CreateTablet for tablet f55b37aa5102423ebbccef3f872d85e8 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260812 19:06:19.342779 29316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe3191acdfe44a019be79eda30b9e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.351346 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.354024 29650 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:19.352617 29649 ts_tablet_manager.cc:1428] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.053s user 0.009s sys 0.005s
I20260812 19:06:19.367409 29650 ts_tablet_manager.cc:1397] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.312s user 0.040s sys 0.038s
I20260812 19:06:19.367978 29649 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.350108 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514939f267f2450faf798117d94726ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.370231 29433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f55b37aa5102423ebbccef3f872d85e8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 19:06:19.370654 29281 leader_election.cc:341] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.371101 29281 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.357990 29658 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.377408 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:19.378085 29341 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.361016 29566 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4ba354bddd43ec97a890c3bb577330 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260812 19:06:19.362680 29320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514939f267f2450faf798117d94726ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.353503 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.366240 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f55b37aa5102423ebbccef3f872d85e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.386409 29578 raft_consensus.cc:2379] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:19.385591 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4ba354bddd43ec97a890c3bb577330 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260812 19:06:19.387861 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4ba354bddd43ec97a890c3bb577330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.389257 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:06:19.390812 29528 leader_election.cc:341] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.391360 29658 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:19.391526 29562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0adb15c8e8a8443a9c36474d5714f204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.392957 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514939f267f2450faf798117d94726ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.381562 29692 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.382710 29314 tablet_service.cc:1467] Processing CreateTablet for tablet f55b37aa5102423ebbccef3f872d85e8 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260812 19:06:19.394797 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f55b37aa5102423ebbccef3f872d85e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.397897 29652 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:19.398478 29566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4ba354bddd43ec97a890c3bb577330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.399053 29310 tablet_service.cc:1467] Processing CreateTablet for tablet a5640e3977714f6e8ffeb9e65d7c70f0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260812 19:06:19.400543 29310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5640e3977714f6e8ffeb9e65d7c70f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.319718 29438 tablet_service.cc:1467] Processing CreateTablet for tablet fe3191acdfe44a019be79eda30b9e79b (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260812 19:06:19.402432 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 0e4ba354bddd43ec97a890c3bb577330 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260812 19:06:19.418772 29438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe3191acdfe44a019be79eda30b9e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.418980 29312 tablet_service.cc:1467] Processing CreateTablet for tablet 64325c7581194fafac6e6e2b485fbdc2 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260812 19:06:19.419440 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e4ba354bddd43ec97a890c3bb577330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.423062 29649 tablet_bootstrap.cc:654] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.406461 29313 tablet_service.cc:1467] Processing CreateTablet for tablet c7f7d48e270c456b9b7eeab6f3165a89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260812 19:06:19.430756 29313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f7d48e270c456b9b7eeab6f3165a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.430408 29443 tablet_service.cc:1467] Processing CreateTablet for tablet 64325c7581194fafac6e6e2b485fbdc2 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260812 19:06:19.433059 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64325c7581194fafac6e6e2b485fbdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.414382 29554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3cc924748e9a482e8536297e097a5226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.403761 29281 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.409338 29444 tablet_service.cc:1467] Processing CreateTablet for tablet a5640e3977714f6e8ffeb9e65d7c70f0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260812 19:06:19.442353 29444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5640e3977714f6e8ffeb9e65d7c70f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.434297 29312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64325c7581194fafac6e6e2b485fbdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.434893 29559 tablet_service.cc:1467] Processing CreateTablet for tablet a5640e3977714f6e8ffeb9e65d7c70f0 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260812 19:06:19.375228 29650 raft_consensus.cc:348] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.447907 29650 raft_consensus.cc:374] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.448879 29341 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:19.449265 29650 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.450206 29650 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.451012 29560 tablet_service.cc:1467] Processing CreateTablet for tablet 64325c7581194fafac6e6e2b485fbdc2 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260812 19:06:19.452464 29557 tablet_service.cc:1467] Processing CreateTablet for tablet bf300c57bddf44ffb79e424049fe4c52 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260812 19:06:19.455839 29553 tablet_service.cc:1467] Processing CreateTablet for tablet 8ea5787113834ce596fc55df3da0e244 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260812 19:06:19.456632 29435 tablet_service.cc:1467] Processing CreateTablet for tablet c7f7d48e270c456b9b7eeab6f3165a89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260812 19:06:19.463522 29324 tablet_service.cc:1467] Processing CreateTablet for tablet bf300c57bddf44ffb79e424049fe4c52 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260812 19:06:19.464766 29324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf300c57bddf44ffb79e424049fe4c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.466528 29693 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.460003 29650 ts_tablet_manager.cc:1428] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.092s user 0.006s sys 0.000s
I20260812 19:06:19.461503 29437 tablet_service.cc:1467] Processing CreateTablet for tablet bf300c57bddf44ffb79e424049fe4c52 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260812 19:06:19.448895 29559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5640e3977714f6e8ffeb9e65d7c70f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.470520 29435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f7d48e270c456b9b7eeab6f3165a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.472412 29560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64325c7581194fafac6e6e2b485fbdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.469758 29561 tablet_service.cc:1467] Processing CreateTablet for tablet c7f7d48e270c456b9b7eeab6f3165a89 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260812 19:06:19.473697 29561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f7d48e270c456b9b7eeab6f3165a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.466826 29530 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:19.477141 29695 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.477692 29695 raft_consensus.cc:686] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:19.478193 29557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf300c57bddf44ffb79e424049fe4c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.478559 29695 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.479300 29652 tablet_bootstrap.cc:654] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.491408 29442 tablet_service.cc:1467] Processing CreateTablet for tablet 8ea5787113834ce596fc55df3da0e244 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260812 19:06:19.492188 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf300c57bddf44ffb79e424049fe4c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.491408 29436 tablet_service.cc:1467] Processing CreateTablet for tablet 39e2c7b7710843cf96e72ac710028fa9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260812 19:06:19.469667 29307 tablet_service.cc:1467] Processing CreateTablet for tablet 8ea5787113834ce596fc55df3da0e244 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260812 19:06:19.502197 29442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ea5787113834ce596fc55df3da0e244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.505519 29556 tablet_service.cc:1467] Processing CreateTablet for tablet 39e2c7b7710843cf96e72ac710028fa9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260812 19:06:19.502125 29650 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
W20260812 19:06:19.512349 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:19.513675 29580 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:19.500732 29553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ea5787113834ce596fc55df3da0e244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.518507 29399 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:19.519641 29672 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.520104 29672 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:19.520905 29672 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.529148 29558 tablet_service.cc:1467] Processing CreateTablet for tablet 7229b6057eba4aae9e2485a7bf1fcf17 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260812 19:06:19.531929 29315 tablet_service.cc:1467] Processing CreateTablet for tablet 39e2c7b7710843cf96e72ac710028fa9 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260812 19:06:19.533017 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39e2c7b7710843cf96e72ac710028fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.533082 29556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39e2c7b7710843cf96e72ac710028fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.544437 29652 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:19.544937 29652 ts_tablet_manager.cc:1397] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.147s user 0.017s sys 0.023s
I20260812 19:06:19.547649 29652 raft_consensus.cc:348] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.548429 29652 raft_consensus.cc:374] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.548712 29652 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.549502 29652 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.553638 29652 ts_tablet_manager.cc:1428] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 19:06:19.561537 29307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ea5787113834ce596fc55df3da0e244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.564697 29649 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.565167 29649 ts_tablet_manager.cc:1397] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.197s user 0.007s sys 0.009s
I20260812 19:06:19.567623 29649 raft_consensus.cc:348] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.568324 29649 raft_consensus.cc:374] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.568586 29649 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.570504 29179 catalog_manager.cc:5495] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:19.574510 29652 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:19.574874 29650 tablet_bootstrap.cc:654] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.581722 29428 tablet_service.cc:1467] Processing CreateTablet for tablet 7229b6057eba4aae9e2485a7bf1fcf17 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260812 19:06:19.583143 29428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7229b6057eba4aae9e2485a7bf1fcf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.584398 29558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7229b6057eba4aae9e2485a7bf1fcf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.585594 29315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39e2c7b7710843cf96e72ac710028fa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.585645 29649 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.602627 29314 tablet_service.cc:1467] Processing CreateTablet for tablet 7229b6057eba4aae9e2485a7bf1fcf17 (DEFAULT_TABLE table=test-workload [id=ed044727589a44c1a5f2e214f86101ed]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260812 19:06:19.604314 29649 ts_tablet_manager.cc:1428] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.039s user 0.006s sys 0.001s
I20260812 19:06:19.605316 29649 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.610216 29314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7229b6057eba4aae9e2485a7bf1fcf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:19.549636 29178 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:19.629557 29649 tablet_bootstrap.cc:654] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.637842 29650 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:19.638293 29652 tablet_bootstrap.cc:654] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.638374 29650 ts_tablet_manager.cc:1397] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.136s user 0.011s sys 0.002s
I20260812 19:06:19.641288 29650 raft_consensus.cc:348] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.642027 29650 raft_consensus.cc:374] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.642319 29650 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.642974 29650 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.645089 29650 ts_tablet_manager.cc:1428] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260812 19:06:19.646087 29650 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:19.646945 29693 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.647395 29693 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.649322 29693 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.652371 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.653153 29580 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:19.653992 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:19.654209 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:19.655103 29693 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260812 19:06:19.655386 29282 leader_election.cc:341] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.655488 29693 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.655893 29693 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.666887 29649 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.667548 29649 ts_tablet_manager.cc:1397] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.062s user 0.013s sys 0.016s
I20260812 19:06:19.667913 29693 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.670365 29649 raft_consensus.cc:348] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.671135 29649 raft_consensus.cc:374] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.671386 29650 tablet_bootstrap.cc:654] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.671433 29649 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.672240 29649 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.674965 29649 ts_tablet_manager.cc:1428] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260812 19:06:19.676620 29649 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
W20260812 19:06:19.677428 29382 tablet.cc:2369] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 19:06:19.688485 29649 tablet_bootstrap.cc:654] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.704263 29652 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:19.704895 29652 ts_tablet_manager.cc:1397] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.131s user 0.015s sys 0.010s
I20260812 19:06:19.708310 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:19.708106 29693 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:19.712167 29282 leader_election.cc:341] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.711505 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:19.713930 29580 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.732055 29652 raft_consensus.cc:348] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.733244 29652 raft_consensus.cc:374] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.734903 29650 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:19.735901 29650 ts_tablet_manager.cc:1397] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.090s user 0.013s sys 0.006s
I20260812 19:06:19.733808 29652 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.743064 29652 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.744800 29649 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.745570 29649 ts_tablet_manager.cc:1397] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.069s user 0.022s sys 0.024s
I20260812 19:06:19.747591 29652 ts_tablet_manager.cc:1428] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.042s user 0.006s sys 0.009s
I20260812 19:06:19.748996 29652 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:19.751189 29649 raft_consensus.cc:348] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.753520 29649 raft_consensus.cc:374] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.753577 29580 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:19.754158 29649 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.755496 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:19.755424 29649 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.757119 29693 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.757845 29693 raft_consensus.cc:686] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:19.754391 29650 raft_consensus.cc:348] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.759063 29650 raft_consensus.cc:374] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.759485 29650 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.759423 29693 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.761909 29650 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.763669 29672 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.764128 29672 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.769551 29649 ts_tablet_manager.cc:1428] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.023s user 0.005s sys 0.007s
I20260812 19:06:19.771036 29649 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.772137 29650 ts_tablet_manager.cc:1428] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.033s user 0.010s sys 0.003s
I20260812 19:06:19.773896 29650 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:19.778162 29695 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.774236 29676 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.779072 29695 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.779518 29676 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.784049 29672 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.784945 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.778765 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:19.787102 29403 leader_election.cc:341] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.791728 29676 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.798424 29650 tablet_bootstrap.cc:654] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:19.799050 29399 leader_election.cc:341] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.799854 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.802385 29695 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:19.802959 29676 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.804710 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.796747 29652 tablet_bootstrap.cc:654] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.806906 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.822981 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.824256 29342 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
W20260812 19:06:19.826138 29399 leader_election.cc:341] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.826512 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:19.827869 29530 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:19.829109 29528 leader_election.cc:341] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:19.829322 29403 leader_election.cc:341] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.830045 29403 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.831993 29676 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.835927 29695 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:19.836519 29695 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.837177 29695 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.844755 29179 catalog_manager.cc:5495] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:19.847748 29649 tablet_bootstrap.cc:654] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.855080 29652 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:19.858269 29695 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.860267 29650 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:19.861083 29650 ts_tablet_manager.cc:1397] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.088s user 0.016s sys 0.030s
I20260812 19:06:19.862325 29693 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.862720 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:19.863194 29693 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.864746 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:19.865660 29341 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.866173 29652 ts_tablet_manager.cc:1397] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.107s user 0.020s sys 0.029s
I20260812 19:06:19.867190 29693 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.868156 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.870152 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:19.870203 29281 leader_election.cc:341] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.871099 29458 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:19.871860 29652 raft_consensus.cc:348] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:19.872790 29528 leader_election.cc:341] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.872969 29282 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.874100 29650 raft_consensus.cc:348] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.874938 29693 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:19.875341 29650 raft_consensus.cc:374] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.876039 29650 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.874125 29695 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:19.878077 29341 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:19.877420 29650 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.873018 29652 raft_consensus.cc:374] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.875459 29693 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:19.880429 29530 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:19.882440 29695 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.883904 29650 ts_tablet_manager.cc:1428] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.022s user 0.009s sys 0.006s
I20260812 19:06:19.884315 29649 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.884895 29649 ts_tablet_manager.cc:1397] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.114s user 0.015s sys 0.017s
I20260812 19:06:19.884721 29695 raft_consensus.cc:686] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:19.885620 29650 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:19.886600 29695 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.879377 29652 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.890370 29652 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.890914 29649 raft_consensus.cc:348] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.892102 29649 raft_consensus.cc:374] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.892585 29649 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.893757 29649 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.897846 29649 ts_tablet_manager.cc:1428] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.012s user 0.005s sys 0.006s
I20260812 19:06:19.880242 29693 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.899113 29649 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.901309 29650 tablet_bootstrap.cc:654] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.903227 29676 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:19.903995 29676 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:06:19.905184 29652 ts_tablet_manager.cc:1428] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.038s user 0.006s sys 0.007s
I20260812 19:06:19.907904 29652 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:19.908829 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:19.912601 29676 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:19.913023 29649 tablet_bootstrap.cc:654] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.919006 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:19.920296 29341 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:19.926312 29652 tablet_bootstrap.cc:654] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:19.921825 29403 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.927848 29178 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:19.935302 29676 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.963254 29676 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:19.968312 29693 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.974190 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:19.975261 29459 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:19.974372 29693 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:19.974706 29649 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:19.976922 29649 ts_tablet_manager.cc:1397] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.078s user 0.022s sys 0.023s
I20260812 19:06:19.977517 29650 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:19.978232 29650 ts_tablet_manager.cc:1397] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.093s user 0.028s sys 0.018s
I20260812 19:06:19.975415 29652 tablet_bootstrap.cc:492] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:19.979007 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:19.980072 29652 ts_tablet_manager.cc:1397] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.073s user 0.036s sys 0.022s
W20260812 19:06:19.981168 29281 leader_election.cc:341] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:19.982461 29649 raft_consensus.cc:348] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.983214 29649 raft_consensus.cc:374] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.983481 29649 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.984205 29649 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.986228 29652 raft_consensus.cc:348] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.987613 29652 raft_consensus.cc:374] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.987118 29650 raft_consensus.cc:348] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.988211 29652 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.988240 29650 raft_consensus.cc:374] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:19.989105 29650 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:19.989744 29459 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:19.989681 29652 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.990520 29650 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:19.991583 29282 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:19.995172 29649 ts_tablet_manager.cc:1428] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.018s user 0.005s sys 0.004s
I20260812 19:06:19.996080 29649 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:19.997134 29693 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:19.997921 29693 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:19.999228 29693 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.004839 29649 tablet_bootstrap.cc:654] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.012312 29650 ts_tablet_manager.cc:1428] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.033s user 0.007s sys 0.005s
I20260812 19:06:20.013816 29650 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.014066 29652 ts_tablet_manager.cc:1428] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.033s user 0.006s sys 0.007s
I20260812 19:06:20.015692 29652 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.027758 29650 tablet_bootstrap.cc:654] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.030143 29649 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.030754 29649 ts_tablet_manager.cc:1397] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.023s
I20260812 19:06:20.033910 29649 raft_consensus.cc:348] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.034865 29649 raft_consensus.cc:374] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.035195 29649 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.036371 29649 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.037042 29652 tablet_bootstrap.cc:654] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.041806 29676 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.043987 29649 ts_tablet_manager.cc:1428] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260812 19:06:20.045009 29649 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:20.048590 29178 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:20.055997 29695 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.056788 29695 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.060470 29695 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.062734 29649 tablet_bootstrap.cc:654] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.062958 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.063107 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.063968 29652 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.064179 29341 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:20.064594 29652 ts_tablet_manager.cc:1397] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.020s
W20260812 19:06:20.065338 29528 leader_election.cc:341] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.066648 29676 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.067322 29676 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.070164 29652 raft_consensus.cc:348] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.070853 29652 raft_consensus.cc:374] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.070319 29530 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:20.071125 29652 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.071521 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:20.075610 29403 leader_election.cc:341] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.075704 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.072410 29676 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.071791 29652 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.078814 29652 ts_tablet_manager.cc:1428] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.014s user 0.005s sys 0.004s
I20260812 19:06:20.079865 29652 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.081470 29695 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.082103 29695 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.082716 29695 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.085285 29658 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260812 19:06:20.077625 29399 leader_election.cc:341] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.088049 29399 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.103600 29652 tablet_bootstrap.cc:654] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.120563 29672 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:20.128496 29693 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.130790 29658 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.131815 29658 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.129395 29693 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:20.136382 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 58822us
I20260812 19:06:20.141460 29658 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.143502 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.144488 29649 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.144922 29649 ts_tablet_manager.cc:1397] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.100s user 0.015s sys 0.011s
W20260812 19:06:20.145815 29282 leader_election.cc:341] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.147156 29695 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.148339 29649 raft_consensus.cc:348] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.149042 29649 raft_consensus.cc:374] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.149290 29649 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.142763 29650 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.150230 29650 ts_tablet_manager.cc:1397] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.137s user 0.025s sys 0.028s
I20260812 19:06:20.142706 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.155928 29693 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.159600 29695 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.160068 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.161093 29578 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:20.162314 29649 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.166446 29649 ts_tablet_manager.cc:1428] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.021s user 0.002s sys 0.006s
W20260812 19:06:20.169543 29530 leader_election.cc:341] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:20.170071 29528 consensus_peers.cc:487] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:20.174335 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:20.174503 29339 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.176079 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:20.178052 29693 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.178668 29693 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.179212 29693 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.165239 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.165148 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:20.183849 29342 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.191859 29528 leader_election.cc:341] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.187033 29650 raft_consensus.cc:348] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.197522 29650 raft_consensus.cc:374] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260812 19:06:20.193511 29528 leader_election.cc:341] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.197988 29650 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.196702 29652 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.198400 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:20.202981 29652 ts_tablet_manager.cc:1397] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.123s user 0.013s sys 0.013s
I20260812 19:06:20.203986 29658 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:20.209609 29652 raft_consensus.cc:348] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.210542 29652 raft_consensus.cc:374] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.210814 29652 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.211483 29652 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.222872 29650 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.223269 29693 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.227913 29650 ts_tablet_manager.cc:1428] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.077s user 0.004s sys 0.011s
I20260812 19:06:20.227917 29342 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:20.228674 29649 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:20.229053 29652 ts_tablet_manager.cc:1428] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.025s user 0.006s sys 0.009s
I20260812 19:06:20.230123 29652 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.230964 29693 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.230041 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:06:20.235180 29282 leader_election.cc:341] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.234200 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:20.236905 29580 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.240991 29649 tablet_bootstrap.cc:654] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.242228 29530 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:20.243849 29650 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.248746 29695 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:20.249691 29695 raft_consensus.cc:686] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:20.251256 29695 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.262952 29672 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.264545 29650 tablet_bootstrap.cc:654] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.267606 29580 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:20.269214 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:20.273123 29692 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
W20260812 19:06:20.286907 29282 consensus_peers.cc:487] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:20.303937 29695 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.295382 29692 raft_consensus.cc:686] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:20.307129 29649 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.307880 29649 ts_tablet_manager.cc:1397] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.094s user 0.031s sys 0.012s
I20260812 19:06:20.308306 29652 tablet_bootstrap.cc:654] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.309185 29695 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.307163 29692 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.310065 29695 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.313859 29649 raft_consensus.cc:348] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.314661 29649 raft_consensus.cc:374] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.314932 29649 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.316172 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.316885 29342 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:20.318354 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:20.319238 29658 raft_consensus.cc:2781] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.319628 29658 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.319927 29658 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.324852 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.327296 29177 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
W20260812 19:06:20.328336 29528 leader_election.cc:341] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.330049 29750 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.330646 29750 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.332964 29750 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.332628 29658 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.319254 29695 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.333580 29750 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260812 19:06:20.315702 29649 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.338688 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.348220 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.352057 29652 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.352072 29649 ts_tablet_manager.cc:1428] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.044s user 0.008s sys 0.000s
I20260812 19:06:20.352733 29652 ts_tablet_manager.cc:1397] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.123s user 0.015s sys 0.008s
I20260812 19:06:20.353269 29649 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:20.353360 29650 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.353974 29650 ts_tablet_manager.cc:1397] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.110s user 0.005s sys 0.031s
I20260812 19:06:20.355355 29652 raft_consensus.cc:348] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.356122 29652 raft_consensus.cc:374] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.356456 29652 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.356731 29650 raft_consensus.cc:348] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.357615 29650 raft_consensus.cc:374] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.358124 29650 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.358835 29650 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.357045 29652 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:20.360481 29281 leader_election.cc:341] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.361155 29658 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.362936 29762 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.366273 29650 ts_tablet_manager.cc:1428] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260812 19:06:20.367012 29650 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
W20260812 19:06:20.362778 29282 leader_election.cc:341] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.369256 29658 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.369809 29282 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.370324 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:20.370157 29658 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.371543 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:20.372104 29342 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.372768 29528 leader_election.cc:341] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.373515 29658 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.377928 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.378687 29458 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:20.369287 29652 ts_tablet_manager.cc:1428] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.016s user 0.004s sys 0.001s
I20260812 19:06:20.383728 29652 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.382021 29649 tablet_bootstrap.cc:654] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.386595 29750 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:20.383198 29179 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:20.393677 29528 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:20.382689 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.400199 29762 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.400879 29762 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.401391 29762 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.404831 29342 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
W20260812 19:06:20.406790 29530 leader_election.cc:341] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.422726 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:20.425339 29695 raft_consensus.cc:2781] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:20.426437 29695 raft_consensus.cc:686] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:20.428094 29695 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.439971 29650 tablet_bootstrap.cc:654] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.453446 29652 tablet_bootstrap.cc:654] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.463889 29762 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.468009 29762 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.468638 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:20.469669 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:20.470557 29459 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.470569 29530 leader_election.cc:341] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.486433 29650 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.487039 29650 ts_tablet_manager.cc:1397] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.120s user 0.016s sys 0.011s
I20260812 19:06:20.486564 29649 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.487749 29649 ts_tablet_manager.cc:1397] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.135s user 0.008s sys 0.018s
I20260812 19:06:20.486429 29177 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:20.491149 29762 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.491338 29650 raft_consensus.cc:348] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.492059 29650 raft_consensus.cc:374] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.492357 29650 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.491892 29762 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.493095 29650 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.492195 29459 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:20.500600 29528 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:20.505722 29762 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.506559 29762 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:20.506953 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:20.509914 29530 leader_election.cc:341] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.512141 29762 raft_consensus.cc:686] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:20.513096 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:20.514972 29528 leader_election.cc:341] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.515604 29528 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:20.516566 29695 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:20.518877 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:20.522202 29580 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.521407 29762 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.522739 29650 ts_tablet_manager.cc:1428] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.035s user 0.009s sys 0.001s
I20260812 19:06:20.527375 29650 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.532286 29692 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.535761 29650 tablet_bootstrap.cc:654] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.538045 29649 raft_consensus.cc:348] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.539245 29649 raft_consensus.cc:374] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.539556 29649 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.540287 29649 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.539412 29692 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.542579 29649 ts_tablet_manager.cc:1428] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.055s user 0.024s sys 0.009s
I20260812 19:06:20.542294 29692 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.543664 29649 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:20.543304 29749 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.545825 29692 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.551762 29652 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.552228 29652 ts_tablet_manager.cc:1397] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.169s user 0.021s sys 0.004s
I20260812 19:06:20.554643 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.558857 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.555626 29459 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
W20260812 19:06:20.561791 29281 leader_election.cc:341] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.566190 29652 raft_consensus.cc:348] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.567010 29652 raft_consensus.cc:374] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.567288 29652 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.572360 29649 tablet_bootstrap.cc:654] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.572908 29652 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.574219 29282 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.581642 29750 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.582481 29652 ts_tablet_manager.cc:1428] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.030s user 0.008s sys 0.006s
I20260812 19:06:20.583074 29750 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.583408 29652 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.588584 29750 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.588073 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.588142 29692 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.588891 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.590058 29692 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:06:20.591408 29281 leader_election.cc:341] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.590621 29692 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.591507 29458 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:20.599363 29282 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.604189 29750 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.604897 29750 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.607048 29750 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.609602 29650 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.610351 29650 ts_tablet_manager.cc:1397] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.083s user 0.010s sys 0.008s
I20260812 19:06:20.623862 29650 raft_consensus.cc:348] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.624822 29650 raft_consensus.cc:374] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.625157 29650 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.625982 29650 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.628633 29650 ts_tablet_manager.cc:1428] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.018s user 0.006s sys 0.002s
I20260812 19:06:20.631356 29650 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.646432 29649 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.646934 29649 ts_tablet_manager.cc:1397] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.103s user 0.013s sys 0.033s
I20260812 19:06:20.647156 29692 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.646358 29652 tablet_bootstrap.cc:654] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.649230 29750 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.647876 29177 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:20.651628 29750 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.653630 29692 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.654834 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:20.655622 29650 tablet_bootstrap.cc:654] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.655508 29458 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.655763 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:20.658244 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:20.660043 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:20.660704 29459 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.662249 29281 leader_election.cc:341] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:20.663050 29281 leader_election.cc:341] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.664508 29458 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:20.665834 29649 raft_consensus.cc:348] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.666543 29649 raft_consensus.cc:374] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.666584 29282 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.671957 29692 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:20.672494 29692 raft_consensus.cc:686] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:20.666870 29649 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.675520 29649 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.677616 29652 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.678191 29652 ts_tablet_manager.cc:1397] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.095s user 0.010s sys 0.016s
I20260812 19:06:20.680084 29650 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.680509 29650 ts_tablet_manager.cc:1397] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.011s
I20260812 19:06:20.680362 29652 raft_consensus.cc:348] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.680886 29652 raft_consensus.cc:374] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.681095 29652 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.677663 29649 ts_tablet_manager.cc:1428] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260812 19:06:20.681332 29692 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.687985 29459 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:20.681627 29652 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.691115 29282 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.692548 29750 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:20.693363 29750 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:20.695024 29650 raft_consensus.cc:348] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.696023 29650 raft_consensus.cc:374] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.696358 29650 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.697124 29650 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.696888 29750 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.699867 29650 ts_tablet_manager.cc:1428] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.019s user 0.008s sys 0.004s
I20260812 19:06:20.691530 29652 ts_tablet_manager.cc:1428] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.013s user 0.004s sys 0.002s
I20260812 19:06:20.692900 29649 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:20.702435 29650 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.706346 29652 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.725200 29459 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.732323 29528 consensus_peers.cc:487] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:20.740561 29650 tablet_bootstrap.cc:654] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.743528 29649 tablet_bootstrap.cc:654] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.740018 29179 catalog_manager.cc:5495] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:20.748359 29652 tablet_bootstrap.cc:654] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.769017 29650 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:20.769522 29650 ts_tablet_manager.cc:1397] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.067s user 0.017s sys 0.014s
I20260812 19:06:20.773192 29650 raft_consensus.cc:348] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.773882 29650 raft_consensus.cc:374] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.774176 29650 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.774055 29762 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.774907 29650 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.776978 29650 ts_tablet_manager.cc:1428] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 19:06:20.777870 29650 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:20.777531 29693 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.780932 29750 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.789163 29750 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.798520 29762 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.799734 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:20.802248 29282 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.799463 29762 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.805694 29458 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.808415 29672 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.809176 29672 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.819126 29179 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:20.825335 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.829506 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.840282 29457 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:20.813382 29762 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:20.866505 29695 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:20.869685 29695 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.870698 29652 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:20.872510 29652 ts_tablet_manager.cc:1397] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.166s user 0.011s sys 0.023s
I20260812 19:06:20.873916 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:20.871793 29798 log.cc:924] Time spent T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.074s user 0.000s sys 0.001s
I20260812 19:06:20.875236 29695 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:20.876107 29281 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.871755 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.876857 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:20.877169 29339 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:20.878325 29693 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:20.876076 29530 leader_election.cc:341] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:20.878988 29403 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:20.872044 29750 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.882170 29652 raft_consensus.cc:348] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.882287 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:20.886066 29652 raft_consensus.cc:374] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.886513 29652 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.887401 29652 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.871225 29672 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:20.885715 29649 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:20.890535 29649 ts_tablet_manager.cc:1397] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.198s user 0.016s sys 0.020s
I20260812 19:06:20.892899 29652 ts_tablet_manager.cc:1428] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.020s user 0.004s sys 0.004s
I20260812 19:06:20.895989 29650 tablet_bootstrap.cc:654] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:20.897001 29652 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:20.896659 29528 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:20.909757 29695 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.914731 29695 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.915450 29695 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.963786 29801 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:20.964596 29801 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:20.964986 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:20.965337 29801 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:20.969579 29530 consensus_peers.cc:487] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:20.973181 29750 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:20.978436 29649 raft_consensus.cc:348] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:20.979087 29649 raft_consensus.cc:374] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:20.979341 29649 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:20.944361 29456 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:20.941064 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:20.990242 29762 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:21.000034 29649 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.001132 29762 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.002368 29649 ts_tablet_manager.cc:1428] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.112s user 0.006s sys 0.006s
I20260812 19:06:21.003399 29649 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:21.001838 29762 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.018819 29695 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.021972 29762 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:21.027235 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.027928 29457 raft_consensus.cc:2379] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:21.030480 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:21.033025 29695 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:21.054883 29652 tablet_bootstrap.cc:654] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:21.057264 29530 leader_election.cc:341] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.088701 29649 tablet_bootstrap.cc:654] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.079632 29801 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.114578 29809 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.115236 29809 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.117547 29809 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.120877 29801 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.121462 29580 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.138250 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:21.138957 29342 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.138163 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.139585 29339 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:21.138211 29579 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.148052 29403 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:21.149729 29820 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:21.173772 29650 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:21.174460 29650 ts_tablet_manager.cc:1397] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.397s user 0.040s sys 0.065s
I20260812 19:06:21.150581 29820 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:21.178444 29820 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:21.180133 29399 leader_election.cc:341] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.189549 29652 tablet_bootstrap.cc:492] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:21.190148 29652 ts_tablet_manager.cc:1397] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.293s user 0.022s sys 0.034s
I20260812 19:06:21.187192 29672 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:21.191308 29528 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:21.190587 29672 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.196743 29672 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.252081 29650 raft_consensus.cc:348] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.252720 29650 raft_consensus.cc:374] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.252945 29650 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.254619 29650 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.262252 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:21.262598 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:21.264034 29459 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:21.269578 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:21.269766 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:21.270475 29458 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.271909 29695 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:21.278476 29650 ts_tablet_manager.cc:1428] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.104s user 0.008s sys 0.000s
I20260812 19:06:21.279613 29652 raft_consensus.cc:348] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.279904 29650 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:21.280905 29652 raft_consensus.cc:374] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.281227 29652 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.281070 29693 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:21.281847 29652 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.284693 29652 ts_tablet_manager.cc:1428] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.094s user 0.007s sys 0.000s
I20260812 19:06:21.285830 29652 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:21.283641 29805 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.287220 29805 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.288408 29649 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:21.288825 29649 ts_tablet_manager.cc:1397] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.286s user 0.031s sys 0.048s
I20260812 19:06:21.290513 29805 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.291082 29692 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:21.291231 29649 raft_consensus.cc:348] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.291884 29649 raft_consensus.cc:374] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.292142 29649 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.292718 29649 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.292833 29692 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:21.296346 29342 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:21.296609 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:21.300165 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.301805 29528 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:06:21.302898 29399 leader_election.cc:341] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:21.303855 29399 leader_election.cc:341] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.303719 29695 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:21.306798 29650 tablet_bootstrap.cc:654] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.311173 29649 ts_tablet_manager.cc:1428] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260812 19:06:21.314052 29649 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:21.314782 29403 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.316493 29672 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:21.316113 29820 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.317337 29672 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:21.318912 29672 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.310653 29458 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:21.335100 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:21.335672 29692 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260812 19:06:21.343385 29695 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260812 19:06:21.363909 29403 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.365859 29805 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:21.366093 29762 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:21.371132 29456 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.393177 29820 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.427695 29649 tablet_bootstrap.cc:654] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.437851 29650 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:21.438370 29650 ts_tablet_manager.cc:1397] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.159s user 0.021s sys 0.036s
I20260812 19:06:21.442432 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:21.442994 29650 raft_consensus.cc:348] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.443714 29650 raft_consensus.cc:374] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.443965 29650 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.444607 29650 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.450839 29652 tablet_bootstrap.cc:654] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.417377 29528 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:21.453585 29650 ts_tablet_manager.cc:1428] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260812 19:06:21.456179 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.462574 29650 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:21.467054 29695 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
W20260812 19:06:21.467278 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 52860.2us
I20260812 19:06:21.467978 29695 raft_consensus.cc:686] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:21.438414 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:21.469432 29695 consensus_queue.cc:237] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:21.477945 29399 leader_election.cc:341] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.452143 29342 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:21.488626 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44959.7us
I20260812 19:06:21.511763 29403 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.519312 29820 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:21.520181 29820 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.526134 29342 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:21.544171 29845 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.544693 29820 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.545055 29845 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.544771 29805 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:21.586299 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:21.604298 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 96626.2us
I20260812 19:06:21.610085 29762 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.610872 29762 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.621109 29403 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.619767 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:21.623622 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.615468 29762 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:21.622332 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.634703 29805 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:21.639292 29341 raft_consensus.cc:2379] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:21.657081 29845 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:21.666692 29847 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260812 19:06:21.667986 29650 tablet_bootstrap.cc:654] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.668272 29649 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:21.668716 29649 ts_tablet_manager.cc:1397] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.355s user 0.027s sys 0.006s
I20260812 19:06:21.676918 29805 raft_consensus.cc:686] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:21.671082 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:21.709673 29453 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:06:21.706796 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 85857.4us
I20260812 19:06:21.711544 29403 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:06:21.712527 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:21.709841 29805 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.713099 29672 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:21.716230 29649 raft_consensus.cc:348] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.717756 29649 raft_consensus.cc:374] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.718067 29649 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.718690 29649 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.669032 29530 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:21.721701 29649 ts_tablet_manager.cc:1428] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.053s user 0.009s sys 0.002s
I20260812 19:06:21.722728 29649 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:21.727807 29652 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:21.727469 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:21.728291 29652 ts_tablet_manager.cc:1397] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.443s user 0.013s sys 0.020s
I20260812 19:06:21.738255 29652 raft_consensus.cc:348] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.739074 29652 raft_consensus.cc:374] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.739334 29652 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.739780 29757 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.740089 29652 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:21.746364 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 77526.1us
I20260812 19:06:21.753244 29652 ts_tablet_manager.cc:1428] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.025s user 0.009s sys 0.002s
I20260812 19:06:21.754333 29652 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:21.757692 29649 tablet_bootstrap.cc:654] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.749307 29281 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.764163 29839 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.770928 29652 tablet_bootstrap.cc:654] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.773708 29179 catalog_manager.cc:5495] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:21.774674 29757 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:21.788703 29839 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:21.790164 29456 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.799059 29650 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:21.799646 29650 ts_tablet_manager.cc:1397] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.337s user 0.054s sys 0.061s
I20260812 19:06:21.804409 29650 raft_consensus.cc:348] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.805147 29650 raft_consensus.cc:374] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.805405 29650 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.806123 29650 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.811290 29650 ts_tablet_manager.cc:1428] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260812 19:06:21.814046 29650 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:21.825932 29652 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:21.826592 29652 ts_tablet_manager.cc:1397] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.072s user 0.009s sys 0.042s
I20260812 19:06:21.828732 29649 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:21.829418 29649 ts_tablet_manager.cc:1397] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.107s user 0.015s sys 0.033s
I20260812 19:06:21.832126 29652 raft_consensus.cc:348] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.832904 29652 raft_consensus.cc:374] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.833175 29652 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.835273 29652 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.836650 29649 raft_consensus.cc:348] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.837340 29652 ts_tablet_manager.cc:1428] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.010s user 0.001s sys 0.005s
I20260812 19:06:21.838147 29649 raft_consensus.cc:374] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.838447 29649 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.838902 29652 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:21.839097 29649 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.837822 29650 tablet_bootstrap.cc:654] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.842192 29649 ts_tablet_manager.cc:1428] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.012s user 0.004s sys 0.004s
I20260812 19:06:21.843072 29649 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:21.853065 29649 tablet_bootstrap.cc:654] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.856172 29652 tablet_bootstrap.cc:654] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.856721 29805 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:21.865294 29650 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:21.865800 29650 ts_tablet_manager.cc:1397] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.009s
I20260812 19:06:21.872604 29649 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:21.873077 29649 ts_tablet_manager.cc:1397] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.030s user 0.007s sys 0.011s
I20260812 19:06:21.873688 29805 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.874269 29805 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.876451 29805 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.877331 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.877991 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:21.878844 29577 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
W20260812 19:06:21.878937 29403 leader_election.cc:341] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.880400 29399 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:21.885030 29805 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:21.885628 29805 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:21.886401 29805 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.886183 29762 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:21.875375 29650 raft_consensus.cc:348] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.889832 29650 raft_consensus.cc:374] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.890163 29650 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.890787 29650 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.895032 29650 ts_tablet_manager.cc:1428] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260812 19:06:21.892395 29649 raft_consensus.cc:348] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.896636 29649 raft_consensus.cc:374] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.896919 29649 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.897589 29649 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.895936 29650 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:21.902009 29649 ts_tablet_manager.cc:1428] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260812 19:06:21.913446 29649 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:21.925477 29672 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.926852 29652 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:21.926359 29672 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.927332 29652 ts_tablet_manager.cc:1397] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.089s user 0.024s sys 0.019s
I20260812 19:06:21.930802 29652 raft_consensus.cc:348] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.931589 29652 raft_consensus.cc:374] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:21.931567 29839 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:21.931980 29652 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:21.932044 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:21.932660 29652 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:21.934299 29403 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.932359 29839 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.934852 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:21.937598 29399 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.937885 29839 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.938412 29399 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:21.942433 29652 ts_tablet_manager.cc:1428] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260812 19:06:21.942703 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:21.943503 29652 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:21.938684 29672 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:21.942703 29823 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:21.946798 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:21.946743 29282 leader_election.cc:341] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:21.947934 29577 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:21.952075 29281 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:21.953593 29839 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:21.954239 29839 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:21.954857 29839 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:21.958281 29649 tablet_bootstrap.cc:654] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:21.951887 29672 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260812 19:06:21.970335 29177 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:06:21.993727 29695 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:21.996961 29805 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:21.998165 29652 tablet_bootstrap.cc:654] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.019776 29805 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.022907 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:22.029481 29650 tablet_bootstrap.cc:654] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.035729 29839 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.038565 29339 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.049173 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:22.050778 29577 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.056640 29649 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:22.057037 29649 ts_tablet_manager.cc:1397] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.144s user 0.033s sys 0.021s
I20260812 19:06:22.059147 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:06:22.064199 29282 leader_election.cc:341] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.074295 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:22.076934 29339 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:22.078305 29649 raft_consensus.cc:348] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.078469 29403 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:22.079077 29649 raft_consensus.cc:374] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.079506 29649 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.079985 29805 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:22.080324 29649 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.080865 29805 raft_consensus.cc:686] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:22.083158 29649 ts_tablet_manager.cc:1428] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.026s user 0.002s sys 0.006s
I20260812 19:06:22.082479 29805 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.084128 29649 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:22.085129 29757 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.086316 29579 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.085932 29757 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.090502 29757 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.097223 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:22.092886 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.098549 29342 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
W20260812 19:06:22.099074 29528 leader_election.cc:341] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.074579 29839 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.084757 29577 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:22.111158 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:22.112920 29757 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:22.113590 29757 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:22.123796 29757 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.141589 29652 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:22.142088 29652 ts_tablet_manager.cc:1397] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.199s user 0.013s sys 0.029s
I20260812 19:06:22.142861 29649 tablet_bootstrap.cc:654] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.147043 29179 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:22.161645 29650 tablet_bootstrap.cc:492] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:22.162166 29650 ts_tablet_manager.cc:1397] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.266s user 0.007s sys 0.024s
I20260812 19:06:22.164086 29652 raft_consensus.cc:348] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.164774 29652 raft_consensus.cc:374] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.165030 29652 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.165751 29652 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.168043 29652 ts_tablet_manager.cc:1428] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.026s user 0.007s sys 0.001s
I20260812 19:06:22.168432 29650 raft_consensus.cc:348] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.169131 29650 raft_consensus.cc:374] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.169381 29650 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.170081 29650 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.171432 29652 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:22.173552 29650 ts_tablet_manager.cc:1428] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260812 19:06:22.174501 29650 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:22.159068 29579 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:22.184914 29577 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.188488 29757 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.197875 29650 tablet_bootstrap.cc:654] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.197052 29805 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:22.199373 29281 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:22.200848 29839 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:22.201632 29839 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:22.203477 29649 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:22.203940 29649 ts_tablet_manager.cc:1397] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.120s user 0.011s sys 0.012s
I20260812 19:06:22.203263 29839 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.218158 29805 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.218955 29805 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.221263 29649 raft_consensus.cc:348] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.222246 29649 raft_consensus.cc:374] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.222534 29649 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.223070 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.223287 29649 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.223793 29341 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:22.225603 29649 ts_tablet_manager.cc:1428] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.021s user 0.004s sys 0.002s
I20260812 19:06:22.225733 29403 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:22.228195 29880 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:22.228618 29880 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:22.228960 29880 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.227967 29652 tablet_bootstrap.cc:654] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.229909 29693 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.230701 29649 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:22.230680 29693 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.235898 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:22.237020 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:22.240612 29805 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.242565 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:22.243937 29577 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:22.244010 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:22.244908 29341 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.248351 29282 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:22.248981 29693 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.249645 29845 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:22.277778 29176 catalog_manager.cc:5495] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:22.296707 29339 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.319136 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:22.304180 29658 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.344923 29880 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.351603 29458 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:22.313184 29757 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.339416 29762 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:22.362244 29762 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.304396 29879 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260812 19:06:22.334293 29658 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.401677 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:22.404055 29658 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.422447 29762 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.423308 29762 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.423843 29762 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.424132 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:22.431059 29528 leader_election.cc:341] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.433876 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:22.432639 29880 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.440834 29341 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:22.444046 29762 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.447009 29649 tablet_bootstrap.cc:700] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent opening tablet: real 0.137s user 0.008s sys 0.033s
I20260812 19:06:22.450634 29649 tablet_bootstrap.cc:654] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.453400 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:22.453805 29652 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:22.457667 29652 ts_tablet_manager.cc:1397] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.286s user 0.015s sys 0.013s
I20260812 19:06:22.451993 29762 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.458443 29762 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:22.458847 29528 leader_election.cc:341] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.461592 29762 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:22.461928 29650 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:22.462898 29650 ts_tablet_manager.cc:1397] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.289s user 0.034s sys 0.082s
I20260812 19:06:22.477237 29650 raft_consensus.cc:348] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.481078 29650 raft_consensus.cc:374] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.481448 29650 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.482515 29650 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.471863 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:22.496042 29650 ts_tablet_manager.cc:1428] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.033s user 0.008s sys 0.001s
I20260812 19:06:22.497733 29650 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:22.504164 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:22.508914 29528 leader_election.cc:341] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.511085 29579 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:06:22.499825 29892 log.cc:924] Time spent T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.052s user 0.000s sys 0.002s
W20260812 19:06:22.485922 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 176716us
I20260812 19:06:22.517568 29880 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.526057 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:22.525635 29880 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.527014 29879 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:06:22.524816 29652 raft_consensus.cc:348] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.530773 29652 raft_consensus.cc:374] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.524801 29672 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260812 19:06:22.532944 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:22.535301 29907 log.cc:924] Time spent T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260812 19:06:22.537163 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:22.531275 29652 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.529172 29880 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.549743 29652 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.553272 29652 ts_tablet_manager.cc:1428] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.090s user 0.017s sys 0.001s
I20260812 19:06:22.554649 29652 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
W20260812 19:06:22.537245 29403 leader_election.cc:341] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.567433 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.538645 29579 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:22.567310 29176 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:22.591428 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:22.598311 29880 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:22.599107 29880 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:22.599643 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.600826 29880 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.610805 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:22.612349 29650 tablet_bootstrap.cc:654] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.609323 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.620759 29906 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:22.611716 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:22.625809 29906 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:22.636595 29906 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.626237 29338 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:22.635684 29879 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.050s
I20260812 19:06:22.699831 29880 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:22.711230 29530 leader_election.cc:341] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.714932 29530 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:22.714229 29652 tablet_bootstrap.cc:654] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:22.719839 29898 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:22.737697 29693 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:22.755317 29880 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.807082 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:22.809623 29649 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:22.813158 29649 ts_tablet_manager.cc:1397] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.583s user 0.015s sys 0.054s
W20260812 19:06:22.820022 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:22.832031 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:22.833824 29580 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.801069 29530 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:22.839634 29906 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:22.841758 29530 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:22.846985 29649 raft_consensus.cc:348] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.849107 29649 raft_consensus.cc:374] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:22.850194 29649 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:22.851353 29649 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:22.858566 29403 leader_election.cc:341] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.855899 29695 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:22.860345 29695 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:22.861313 29695 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:22.867074 29649 ts_tablet_manager.cc:1428] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.053s user 0.011s sys 0.000s
I20260812 19:06:22.868628 29649 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:22.926680 29179 catalog_manager.cc:5495] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
W20260812 19:06:22.943854 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 102416us
I20260812 19:06:22.957432 29580 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:22.944773 29652 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
W20260812 19:06:22.946175 29282 consensus_peers.cc:487] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:22.952833 29650 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:22.963621 29650 ts_tablet_manager.cc:1397] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.467s user 0.027s sys 0.056s
I20260812 19:06:22.962299 29652 ts_tablet_manager.cc:1397] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.408s user 0.007s sys 0.038s
I20260812 19:06:22.970177 29845 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:22.971040 29845 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.978204 29649 tablet_bootstrap.cc:654] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:22.962096 29530 leader_election.cc:341] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.983366 29530 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:22.985733 29845 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:22.986959 29898 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:22.960506 29695 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:22.969936 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:22.989327 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:22.996237 29282 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:22.996294 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:22.999725 29880 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:23.001111 29880 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:23.002575 29880 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.005249 29695 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:23.010115 29281 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.010668 29652 raft_consensus.cc:348] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.012128 29652 raft_consensus.cc:374] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.012625 29652 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.012918 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:23.013676 29652 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.016022 29695 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.016121 29650 raft_consensus.cc:348] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.019670 29650 raft_consensus.cc:374] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.020207 29650 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.017081 29695 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.025319 29650 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.011049 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:23.044400 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:23.044828 29845 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.045928 29338 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:23.053157 29528 leader_election.cc:341] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.059815 29652 ts_tablet_manager.cc:1428] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.095s user 0.023s sys 0.024s
I20260812 19:06:23.060909 29650 ts_tablet_manager.cc:1428] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.096s user 0.006s sys 0.006s
I20260812 19:06:23.061249 29652 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:23.066563 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:23.073483 29650 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:23.056663 29695 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:23.079752 29649 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:23.080747 29649 ts_tablet_manager.cc:1397] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.212s user 0.021s sys 0.010s
W20260812 19:06:23.089505 29530 leader_election.cc:341] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.095073 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:23.112142 29649 raft_consensus.cc:348] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.118695 29338 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:23.121323 29649 raft_consensus.cc:374] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.119882 29652 tablet_bootstrap.cc:654] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.122035 29649 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
W20260812 19:06:23.119755 29528 leader_election.cc:341] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.124707 29528 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:23.127866 29906 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.132743 29650 tablet_bootstrap.cc:654] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.133392 29898 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:23.128329 29178 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:23.136976 29530 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:23.124176 29649 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.140365 29652 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:23.140789 29906 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:23.141191 29652 ts_tablet_manager.cc:1397] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.080s user 0.015s sys 0.007s
I20260812 19:06:23.141745 29906 raft_consensus.cc:686] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:23.144976 29649 ts_tablet_manager.cc:1428] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.064s user 0.010s sys 0.001s
I20260812 19:06:23.144613 29906 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.146708 29649 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:23.146881 29652 raft_consensus.cc:348] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.148080 29652 raft_consensus.cc:374] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.148530 29652 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.149572 29652 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.153833 29652 ts_tablet_manager.cc:1428] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260812 19:06:23.155407 29652 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:23.161940 29930 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.162864 29930 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.165191 29650 tablet_bootstrap.cc:492] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:23.166814 29930 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.168624 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:23.169997 29579 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:23.174263 29695 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:23.178932 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:23.183377 29650 ts_tablet_manager.cc:1397] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.110s user 0.017s sys 0.003s
I20260812 19:06:23.198917 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:23.201766 29931 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:23.208551 29880 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.211720 29880 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.218704 29652 tablet_bootstrap.cc:654] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.239452 29880 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.241583 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:23.244153 29337 raft_consensus.cc:2379] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:23.246805 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:23.255146 29399 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:23.264360 29940 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.265789 29940 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.259908 29880 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:23.279896 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:23.284649 29530 leader_election.cc:341] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.294202 29649 tablet_bootstrap.cc:654] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.296696 29931 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.297719 29940 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:23.286098 29650 raft_consensus.cc:348] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.299052 29650 raft_consensus.cc:374] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.299333 29650 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.300012 29650 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.303921 29650 ts_tablet_manager.cc:1428] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.120s user 0.005s sys 0.004s
I20260812 19:06:23.305708 29650 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:23.305833 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:23.309712 29528 leader_election.cc:341] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.311234 29528 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:23.311975 29931 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.313117 29931 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.316839 29940 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.335428 29931 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.333519 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:23.419715 29282 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.430737 29880 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.330112 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:23.447383 29652 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:23.448360 29652 ts_tablet_manager.cc:1397] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.293s user 0.022s sys 0.017s
W20260812 19:06:23.459611 29281 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.465111 29281 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:23.466399 29650 tablet_bootstrap.cc:654] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.471742 29931 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.485430 29652 raft_consensus.cc:348] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.489728 29652 raft_consensus.cc:374] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.490260 29652 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.491387 29652 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.495726 29652 ts_tablet_manager.cc:1428] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.047s user 0.013s sys 0.000s
I20260812 19:06:23.512684 29652 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:23.520792 29880 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:23.546388 29649 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:23.547377 29649 ts_tablet_manager.cc:1397] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.401s user 0.013s sys 0.035s
I20260812 19:06:23.549362 29176 catalog_manager.cc:5495] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:23.556145 29649 raft_consensus.cc:348] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.559434 29880 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:23.557595 29649 raft_consensus.cc:374] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.561901 29649 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.563691 29649 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.575181 29652 tablet_bootstrap.cc:654] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.586305 29649 ts_tablet_manager.cc:1428] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.038s user 0.006s sys 0.015s
I20260812 19:06:23.588151 29649 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:23.591696 29650 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:23.592772 29650 ts_tablet_manager.cc:1397] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.287s user 0.036s sys 0.047s
I20260812 19:06:23.596771 29898 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:23.626582 29456 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:23.642902 29650 raft_consensus.cc:348] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.644374 29650 raft_consensus.cc:374] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.644838 29650 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.651397 29650 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.657117 29650 ts_tablet_manager.cc:1428] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.055s user 0.009s sys 0.001s
I20260812 19:06:23.658588 29650 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:23.671989 29880 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.672966 29652 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:23.672974 29880 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.673924 29652 ts_tablet_manager.cc:1397] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.163s user 0.017s sys 0.019s
I20260812 19:06:23.677103 29880 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.681133 29845 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.685557 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:23.687273 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:23.688587 29906 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260812 19:06:23.694377 29399 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.682322 29845 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:23.702667 29403 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.703980 29403 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:23.710285 29652 raft_consensus.cc:348] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.711589 29652 raft_consensus.cc:374] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.712172 29652 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.713157 29652 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.718225 29652 ts_tablet_manager.cc:1428] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.043s user 0.009s sys 0.003s
I20260812 19:06:23.719890 29652 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:23.731506 29649 tablet_bootstrap.cc:654] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.745033 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:23.723109 29943 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:23.740348 29845 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.854573 29652 tablet_bootstrap.cc:654] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.812603 29950 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:23.866358 29950 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.846289 29579 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:23.871052 29650 tablet_bootstrap.cc:654] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:23.739149 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:23.871814 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:23.874583 29930 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:23.875306 29930 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:23.876045 29930 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:23.895790 29950 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.897855 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:23.907038 29399 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:23.961052 29649 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:23.962829 29930 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.962054 29649 ts_tablet_manager.cc:1397] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.374s user 0.016s sys 0.029s
I20260812 19:06:23.970589 29649 raft_consensus.cc:348] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.971745 29649 raft_consensus.cc:374] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:23.972211 29649 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:23.973306 29880 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:23.975214 29880 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.981158 29930 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:23.984972 29652 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:23.985869 29652 ts_tablet_manager.cc:1397] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.266s user 0.020s sys 0.009s
I20260812 19:06:23.988402 29649 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:23.994987 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:23.996284 29580 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:24.001910 29282 leader_election.cc:341] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:24.004631 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:23.995877 29649 ts_tablet_manager.cc:1428] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.031s user 0.005s sys 0.005s
I20260812 19:06:24.009650 29649 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:24.017823 29652 raft_consensus.cc:348] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.016556 29880 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.034487 29652 raft_consensus.cc:374] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.035019 29652 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.035691 29580 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:24.017763 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:24.037638 29337 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:24.018673 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:24.017908 29650 tablet_bootstrap.cc:492] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:24.041791 29650 ts_tablet_manager.cc:1397] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.383s user 0.029s sys 0.048s
I20260812 19:06:24.037878 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:24.047133 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:24.050580 29930 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.051173 29650 raft_consensus.cc:348] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.051915 29930 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:24.052596 29650 raft_consensus.cc:374] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.053170 29650 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.036212 29652 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.057333 29906 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.059041 29906 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.060438 29403 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:24.062469 29880 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.063608 29880 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.064373 29880 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.054621 29930 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.068744 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:24.073062 29953 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:24.064284 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:24.057397 29650 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.067814 29906 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:24.086970 29649 tablet_bootstrap.cc:654] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.091228 29652 ts_tablet_manager.cc:1428] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.102s user 0.010s sys 0.001s
I20260812 19:06:24.091437 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:24.092954 29652 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
W20260812 19:06:24.097417 29282 leader_election.cc:341] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.101874 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.164310 29652 tablet_bootstrap.cc:654] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.146878 29845 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.165150 29456 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:24.165124 29845 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.167711 29528 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:24.163384 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 66087.3us
I20260812 19:06:24.169889 29695 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.170641 29695 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.171244 29695 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.171358 29845 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.174124 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:24.175446 29577 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:24.174962 29880 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.162120 29650 ts_tablet_manager.cc:1428] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.116s user 0.014s sys 0.003s
I20260812 19:06:24.136160 29176 catalog_manager.cc:5495] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:24.140964 29930 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.179052 29281 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:24.180029 29530 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.179492 29930 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.181699 29955 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.182682 29955 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.183301 29955 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.183820 29930 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.189477 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:24.191562 29577 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:24.194309 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:24.196256 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.179078 29650 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:24.196795 29930 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.202816 29930 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.204134 29930 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.199091 29452 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:24.207149 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:06:24.210871 29399 leader_election.cc:341] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.217274 29880 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.229465 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:24.229040 29695 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.231465 29338 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.236747 29695 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:24.241261 29649 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:24.242328 29649 ts_tablet_manager.cc:1397] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.233s user 0.033s sys 0.005s
I20260812 19:06:24.312126 29649 raft_consensus.cc:348] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.315796 29649 raft_consensus.cc:374] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.316282 29649 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.318343 29649 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.257401 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:24.323694 29649 ts_tablet_manager.cc:1428] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.081s user 0.010s sys 0.002s
I20260812 19:06:24.326179 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.327812 29649 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:24.334717 29650 tablet_bootstrap.cc:654] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.293606 29338 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:24.316427 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:24.272500 29930 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.346324 29456 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.369612 29652 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:24.373071 29652 ts_tablet_manager.cc:1397] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.280s user 0.029s sys 0.026s
I20260812 19:06:24.398514 29649 tablet_bootstrap.cc:654] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.396987 29652 raft_consensus.cc:348] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.399844 29652 raft_consensus.cc:374] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.400385 29652 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.401613 29652 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.396538 29930 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.403834 29650 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:24.404734 29650 ts_tablet_manager.cc:1397] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.226s user 0.027s sys 0.058s
I20260812 19:06:24.405414 29652 ts_tablet_manager.cc:1428] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.028s user 0.006s sys 0.002s
I20260812 19:06:24.392299 29955 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.406814 29652 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:24.478794 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:24.482113 29580 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.409873 29955 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:24.546890 29282 leader_election.cc:341] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.556079 29456 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:24.575523 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:24.578084 29403 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:24.578855 29577 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.540774 29650 raft_consensus.cc:348] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:24.582985 29530 leader_election.cc:341] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.588294 29650 raft_consensus.cc:374] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.591912 29528 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:24.593328 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:24.594907 29452 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.596239 29695 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.581358 29950 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.600553 29950 raft_consensus.cc:686] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:24.603428 29695 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:24.602852 29950 consensus_queue.cc:237] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.599094 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:24.605558 29695 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.590379 29650 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.616204 29652 tablet_bootstrap.cc:654] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:24.613381 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46195.5us
I20260812 19:06:24.623482 29580 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:24.627902 29650 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.646268 29650 ts_tablet_manager.cc:1428] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.241s user 0.017s sys 0.001s
I20260812 19:06:24.682901 29650 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:24.715248 29649 tablet_bootstrap.cc:492] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:24.716636 29649 ts_tablet_manager.cc:1397] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.389s user 0.032s sys 0.052s
I20260812 19:06:24.724696 29652 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:24.724946 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:24.726052 29652 ts_tablet_manager.cc:1397] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.319s user 0.048s sys 0.045s
I20260812 19:06:24.737630 29930 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.738798 29930 raft_consensus.cc:686] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:24.740332 29930 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.744961 29980 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260812 19:06:24.752774 29176 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:24.767784 29452 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:24.752424 29177 catalog_manager.cc:5495] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:24.780236 29649 raft_consensus.cc:348] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.781822 29649 raft_consensus.cc:374] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260812 19:06:24.773833 29282 leader_election.cc:341] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.782387 29649 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.790400 29649 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.796351 29649 ts_tablet_manager.cc:1428] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.079s user 0.010s sys 0.002s
I20260812 19:06:24.799572 29987 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.800647 29987 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.791653 29652 raft_consensus.cc:348] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.806042 29652 raft_consensus.cc:374] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.805912 29987 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:24.806743 29652 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.807994 29652 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.810031 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.814330 29980 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:06:24.815007 29528 leader_election.cc:341] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.815156 29980 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.787377 29577 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:24.820194 29282 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:24.821748 29650 tablet_bootstrap.cc:654] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.817677 29980 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.827966 29652 ts_tablet_manager.cc:1428] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.101s user 0.011s sys 0.000s
I20260812 19:06:24.830973 29930 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:24.829845 29652 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:24.843417 29649 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:24.831342 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:24.849131 29577 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:24.832095 29930 raft_consensus.cc:686] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:24.851419 29281 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:24.852058 29930 consensus_queue.cc:237] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.866997 29931 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:24.867686 29931 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:24.868250 29931 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.878618 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.883910 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:24.890579 29282 consensus_peers.cc:487] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260812 19:06:24.903787 29282 leader_election.cc:341] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:24.914018 29530 leader_election.cc:341] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.916689 29530 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:24.918308 29695 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:24.932024 29650 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:24.919055 29931 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.932907 29650 ts_tablet_manager.cc:1397] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.252s user 0.031s sys 0.030s
I20260812 19:06:24.935290 29931 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:24.941216 29650 raft_consensus.cc:348] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.942734 29650 raft_consensus.cc:374] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:24.943338 29650 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:24.944552 29650 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.948290 29649 tablet_bootstrap.cc:654] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.951829 29652 tablet_bootstrap.cc:654] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:24.945478 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:24.957769 29579 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:24.948722 29650 ts_tablet_manager.cc:1428] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
I20260812 19:06:24.955178 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:24.961999 29650 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
W20260812 19:06:24.984191 29282 leader_election.cc:341] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:24.984925 29987 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.985841 29987 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:24.991818 29987 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:24.992003 29652 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:24.992297 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:24.993117 29652 ts_tablet_manager.cc:1397] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.164s user 0.026s sys 0.004s
I20260812 19:06:24.999100 29930 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:24.994027 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.001164 29337 raft_consensus.cc:2379] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:25.000165 29930 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.003005 29530 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:25.005244 29987 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.004287 29178 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:25.007653 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:25.009846 29281 leader_election.cc:341] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.010905 29178 catalog_manager.cc:5495] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:25.016145 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.017191 29452 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:25.019390 29282 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:25.020752 29931 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:25.021391 29931 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:25.021914 29931 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.005295 29930 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.028386 29579 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:25.031042 29281 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:25.034516 29845 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:25.035059 29649 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:25.035841 29649 ts_tablet_manager.cc:1397] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.193s user 0.027s sys 0.017s
I20260812 19:06:25.037091 29845 raft_consensus.cc:686] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:25.051759 29845 consensus_queue.cc:237] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.042271 29650 tablet_bootstrap.cc:654] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.040648 29649 raft_consensus.cc:348] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.057308 29649 raft_consensus.cc:374] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.034014 29652 raft_consensus.cc:348] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.057862 29649 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.058290 29652 raft_consensus.cc:374] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.058822 29652 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.060084 29652 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.044371 29987 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:25.033093 29982 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.062419 29982 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.059073 29649 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.066226 29652 ts_tablet_manager.cc:1428] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.067s user 0.010s sys 0.002s
I20260812 19:06:25.068328 29652 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:25.070339 29950 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.072256 29931 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.077924 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:06:25.080873 29281 leader_election.cc:341] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.080237 29982 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.082337 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:25.086831 29577 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:25.088791 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.084959 29649 ts_tablet_manager.cc:1428] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.049s user 0.014s sys 0.000s
I20260812 19:06:25.090965 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:25.092330 29649 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:25.093034 29982 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:25.085129 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:25.093804 29982 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:25.095144 29982 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:25.096341 29403 leader_election.cc:341] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.094348 29452 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.107124 29931 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.113718 29652 tablet_bootstrap.cc:654] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.132027 29880 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.150233 29649 tablet_bootstrap.cc:654] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.170226 29650 tablet_bootstrap.cc:492] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:25.172291 29650 ts_tablet_manager.cc:1397] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.211s user 0.042s sys 0.050s
I20260812 19:06:25.178206 29339 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.189736 29452 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:25.190254 29177 catalog_manager.cc:5495] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:25.191936 29282 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:25.195313 29845 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:25.205637 29845 raft_consensus.cc:686] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:25.207302 29845 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.212102 29982 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.228919 29906 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.237210 29650 raft_consensus.cc:348] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.238577 29650 raft_consensus.cc:374] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.239034 29650 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.240386 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:25.241587 29577 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.235966 29982 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.246184 29652 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:25.247004 29652 ts_tablet_manager.cc:1397] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.179s user 0.018s sys 0.012s
I20260812 19:06:25.249521 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:25.251379 29650 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.259230 29695 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260812 19:06:25.266176 29650 ts_tablet_manager.cc:1428] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.093s user 0.007s sys 0.005s
W20260812 19:06:25.266728 29403 leader_election.cc:341] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.268036 29650 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:25.272089 29649 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:25.282516 29649 ts_tablet_manager.cc:1397] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.191s user 0.019s sys 0.007s
I20260812 19:06:25.287616 29649 raft_consensus.cc:348] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:25.288475 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:25.289081 29649 raft_consensus.cc:374] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.290464 29652 raft_consensus.cc:348] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.289623 29649 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.291836 29652 raft_consensus.cc:374] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.292517 29652 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.293771 29652 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.298194 29652 ts_tablet_manager.cc:1428] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.051s user 0.014s sys 0.000s
I20260812 19:06:25.300333 29652 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:25.315970 29652 tablet_bootstrap.cc:654] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:25.318434 29282 consensus_peers.cc:487] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:25.318301 29649 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.325932 29649 ts_tablet_manager.cc:1428] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.043s user 0.009s sys 0.001s
I20260812 19:06:25.328770 29650 tablet_bootstrap.cc:654] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.328305 29931 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260812 19:06:25.336549 29649 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:25.337098 29906 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.339691 29880 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.340963 29880 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.345278 29178 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:25.349285 29880 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.353273 29577 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:25.357827 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.360260 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:25.375629 29579 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:25.361742 29906 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.405377 29906 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:25.408123 29652 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:25.410969 29931 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.411867 29652 ts_tablet_manager.cc:1397] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.112s user 0.028s sys 0.025s
I20260812 19:06:25.409068 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:25.417812 29649 tablet_bootstrap.cc:654] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.418987 29982 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:25.419986 29982 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:25.422364 29982 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.426509 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.427906 29337 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:25.425770 29652 raft_consensus.cc:348] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.430203 29652 raft_consensus.cc:374] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.433027 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:25.434969 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:25.430758 29652 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.438571 29898 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:25.439118 29880 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.439410 29898 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:25.440130 29898 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.442814 29652 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.447733 29931 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.451997 29649 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:25.453377 29649 ts_tablet_manager.cc:1397] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.117s user 0.024s sys 0.001s
I20260812 19:06:25.458689 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.459421 29652 ts_tablet_manager.cc:1428] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.047s user 0.013s sys 0.000s
I20260812 19:06:25.461311 29652 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:25.468453 29880 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:25.469249 29650 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:25.470269 29650 ts_tablet_manager.cc:1397] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.203s user 0.019s sys 0.015s
I20260812 19:06:25.481619 29649 raft_consensus.cc:348] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.483698 29649 raft_consensus.cc:374] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.484133 29649 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.485356 29649 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:25.488452 29528 leader_election.cc:341] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.492568 29650 raft_consensus.cc:348] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.493422 29650 raft_consensus.cc:374] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.493709 29650 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.494748 29650 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.499485 29649 ts_tablet_manager.cc:1428] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.045s user 0.010s sys 0.001s
I20260812 19:06:25.501015 29650 ts_tablet_manager.cc:1428] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.030s user 0.009s sys 0.000s
I20260812 19:06:25.503149 29650 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:25.506379 29649 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:25.516959 29652 tablet_bootstrap.cc:654] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.577685 29650 tablet_bootstrap.cc:654] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.610164 30028 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.611517 30028 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.598664 29845 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260812 19:06:25.626713 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:25.615638 30028 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.655761 29649 tablet_bootstrap.cc:654] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.675048 29177 catalog_manager.cc:5495] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
W20260812 19:06:25.681361 29281 leader_election.cc:341] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:25.675922 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 86483.7us
I20260812 19:06:25.682596 29652 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:25.688117 29898 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.690266 29652 ts_tablet_manager.cc:1397] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.229s user 0.036s sys 0.025s
I20260812 19:06:25.693140 29898 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:25.695240 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:25.694967 29652 raft_consensus.cc:348] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.698274 29652 raft_consensus.cc:374] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.698889 29652 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
W20260812 19:06:25.700392 29528 leader_election.cc:341] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.700171 29652 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.707727 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.709081 29898 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.710186 29898 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.704149 29652 ts_tablet_manager.cc:1428] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.013s user 0.010s sys 0.000s
I20260812 19:06:25.715816 29652 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:25.715898 29898 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:25.730736 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.732573 29456 raft_consensus.cc:2379] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:25.732295 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:25.733707 29338 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.747503 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.755307 29649 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:25.756227 29649 ts_tablet_manager.cc:1397] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.250s user 0.020s sys 0.010s
I20260812 19:06:25.760425 29650 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:25.773525 29650 ts_tablet_manager.cc:1397] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.271s user 0.026s sys 0.014s
W20260812 19:06:25.775326 29282 leader_election.cc:341] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.776154 29282 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:25.779559 29650 raft_consensus.cc:348] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.782434 30031 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:25.783274 29650 raft_consensus.cc:374] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.786089 29650 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.784525 30031 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260812 19:06:25.796346 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:25.798235 29906 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.787921 29650 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.803413 30031 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.797925 29649 raft_consensus.cc:348] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.806795 29652 tablet_bootstrap.cc:654] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.810034 29650 ts_tablet_manager.cc:1428] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.036s user 0.011s sys 0.000s
I20260812 19:06:25.811748 29650 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:25.812973 29338 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:25.806798 29649 raft_consensus.cc:374] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.806938 30031 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.815200 30037 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.816824 30031 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.817759 29649 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:25.825629 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:25.832377 29281 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.833701 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.819211 29649 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.833873 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
W20260812 19:06:25.849514 29282 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.850699 29282 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:25.852542 30031 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:25.855472 29898 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:25.856353 29898 raft_consensus.cc:686] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:25.857806 29898 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.875204 29906 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:25.877748 29649 ts_tablet_manager.cc:1428] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.121s user 0.014s sys 0.001s
I20260812 19:06:25.879899 29649 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:25.882583 29953 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.883409 29953 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.889892 29649 tablet_bootstrap.cc:654] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.890005 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.891508 29338 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:25.891981 29953 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.892289 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:25.893527 29577 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:25.893756 29403 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:25.895805 29953 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:25.896512 29953 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:25.897228 29953 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.902334 30031 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:25.916218 29898 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:25.917058 29898 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.924501 29650 tablet_bootstrap.cc:654] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:25.921418 29953 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.927361 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:25.942377 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:25.943874 29338 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.946668 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:25.935496 29898 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:25.960574 29649 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:25.961436 29649 ts_tablet_manager.cc:1397] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.082s user 0.011s sys 0.014s
I20260812 19:06:25.962602 29953 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:25.966325 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:25.967622 29578 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:25.967864 29649 raft_consensus.cc:348] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:25.969292 29649 raft_consensus.cc:374] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:25.969738 29649 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
W20260812 19:06:25.951133 29530 leader_election.cc:341] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:25.981046 29649 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.016117 29695 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.016932 29695 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:26.016026 29403 consensus_peers.cc:487] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260812 19:06:26.013594 29528 leader_election.cc:341] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.028749 29528 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:26.030239 30029 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:26.036188 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:26.051090 29650 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:26.052048 29650 ts_tablet_manager.cc:1397] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.241s user 0.043s sys 0.030s
I20260812 19:06:26.053617 29176 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:26.056437 29695 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:26.055359 29530 leader_election.cc:341] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.074388 29652 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:26.073570 29649 ts_tablet_manager.cc:1428] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.112s user 0.023s sys 0.016s
I20260812 19:06:26.076005 29652 ts_tablet_manager.cc:1397] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.361s user 0.050s sys 0.090s
I20260812 19:06:26.079919 29880 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:26.080916 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:26.084594 29650 raft_consensus.cc:348] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.100631 29650 raft_consensus.cc:374] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.101128 29650 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
W20260812 19:06:26.102622 29528 leader_election.cc:341] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.088683 29338 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:26.103466 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:26.102416 29650 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.104789 29906 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:26.105444 29403 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:26.106179 29652 raft_consensus.cc:348] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.107401 29880 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:26.108371 29880 raft_consensus.cc:686] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:26.108896 29650 ts_tablet_manager.cc:1428] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.056s user 0.010s sys 0.002s
I20260812 19:06:26.109867 29880 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.113487 29650 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:26.092226 29578 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:26.109752 29649 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:26.107620 29652 raft_consensus.cc:374] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.126897 29652 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.131481 29652 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.134630 29652 ts_tablet_manager.cc:1428] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.033s user 0.010s sys 0.000s
I20260812 19:06:26.136322 29652 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:26.138271 29650 tablet_bootstrap.cc:654] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.148968 29649 tablet_bootstrap.cc:654] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.150429 29953 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.151276 29953 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.157080 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:26.160821 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.162016 29650 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:26.162876 29650 ts_tablet_manager.cc:1397] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.050s user 0.016s sys 0.010s
I20260812 19:06:26.163630 29399 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:26.164986 29982 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:26.169935 29650 raft_consensus.cc:348] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.171536 29650 raft_consensus.cc:374] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.171950 29650 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.158993 29953 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.173498 29650 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.178699 29650 ts_tablet_manager.cc:1428] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260812 19:06:26.181517 29650 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:26.192654 29650 tablet_bootstrap.cc:654] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.196280 29649 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:26.197062 29649 ts_tablet_manager.cc:1397] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.088s user 0.020s sys 0.007s
I20260812 19:06:26.202116 29652 tablet_bootstrap.cc:654] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.215651 29650 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:26.216459 29650 ts_tablet_manager.cc:1397] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.014s
I20260812 19:06:26.218384 29880 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.219170 29880 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.220924 29650 raft_consensus.cc:348] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.222204 29650 raft_consensus.cc:374] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.224961 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:26.228441 29338 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:26.222648 29650 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.231455 29650 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.232736 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.235637 29650 ts_tablet_manager.cc:1428] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.019s user 0.013s sys 0.000s
I20260812 19:06:26.237504 29650 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:26.237164 29649 raft_consensus.cc:348] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.238615 29649 raft_consensus.cc:374] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.239080 29649 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.240147 29649 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:26.242610 29399 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.244558 29880 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.254959 29652 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:26.256181 29652 ts_tablet_manager.cc:1397] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.120s user 0.019s sys 0.002s
I20260812 19:06:26.252126 29176 catalog_manager.cc:5495] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:26.259305 29650 tablet_bootstrap.cc:654] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.262424 29403 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:26.264398 29880 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:26.264714 29649 ts_tablet_manager.cc:1428] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.067s user 0.010s sys 0.002s
I20260812 19:06:26.265056 29880 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:26.265815 29880 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.266629 29649 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:26.279156 29652 raft_consensus.cc:348] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.280643 29652 raft_consensus.cc:374] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.281237 29652 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.286240 30066 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.287235 30066 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.296510 29457 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.299965 30066 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.303582 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.306296 29695 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.307195 29695 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.320290 29652 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.320293 29695 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:26.317464 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:26.296569 29953 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.340000 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:26.341710 29339 raft_consensus.cc:2379] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:26.322772 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:26.354816 29528 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.347422 29953 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.368743 30036 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.378983 29652 ts_tablet_manager.cc:1428] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.118s user 0.012s sys 0.001s
I20260812 19:06:26.382790 29453 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.386919 29403 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:26.378297 30036 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:26.420403 29530 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.421806 29530 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:26.409273 29953 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.424465 29650 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:26.425333 29650 ts_tablet_manager.cc:1397] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.188s user 0.039s sys 0.032s
I20260812 19:06:26.424662 30032 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:26.431129 30032 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.434527 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.443197 29650 raft_consensus.cc:348] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.445466 29695 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:26.447245 29650 raft_consensus.cc:374] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:26.448983 29650 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:26.450325 29650 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.452001 29652 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:26.445312 30036 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.465385 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:26.467029 29456 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:26.446077 30029 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:26.475679 30032 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.478969 29282 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:26.437552 29880 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.486407 30032 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:26.487185 30032 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:26.487934 30032 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.496992 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:26.500243 29456 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
W20260812 19:06:26.515488 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 123274us
I20260812 19:06:26.524216 29282 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:26.491315 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:26.526981 29650 ts_tablet_manager.cc:1428] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.101s user 0.011s sys 0.006s
I20260812 19:06:26.552291 30036 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:26.569888 29650 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:26.563529 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:26.574992 29339 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.558964 29880 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:26.560469 29281 leader_election.cc:341] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.585760 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.571830 30036 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:26.596154 29579 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:26.604480 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:26.599275 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:26.611208 30036 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.610283 29953 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:26.629717 29953 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:26.625779 30066 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260812 19:06:26.639292 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:26.647364 29982 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:26.649526 29953 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.670994 29652 tablet_bootstrap.cc:654] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.659091 29577 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:26.651185 30032 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.709620 29339 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
W20260812 19:06:26.684796 30070 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.061s user 0.000s sys 0.001s
W20260812 19:06:26.748564 29399 leader_election.cc:341] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:26.762367 29906 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:26.775652 29649 tablet_bootstrap.cc:654] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:26.783661 29403 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:26.786489 29982 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:26.787418 29982 raft_consensus.cc:686] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:26.792373 29880 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260812 19:06:26.788959 29982 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.804365 30032 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.821725 29650 tablet_bootstrap.cc:654] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:26.851490 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:26.848399 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:26.891184 29452 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:26.894333 29953 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.854403 30036 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:26.906765 29341 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:26.943176 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:26.945559 29451 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:26.972700 29452 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:26.958621 30036 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.952925 29179 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:26.975946 29652 tablet_bootstrap.cc:492] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:27.014547 29652 ts_tablet_manager.cc:1397] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.563s user 0.027s sys 0.035s
I20260812 19:06:26.985718 29953 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:26.991945 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:27.026155 29282 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:27.025611 29577 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:27.007714 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:27.034389 30077 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:27.035363 30077 raft_consensus.cc:686] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:27.037117 30077 consensus_queue.cc:237] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.041893 29652 raft_consensus.cc:348] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.043687 29652 raft_consensus.cc:374] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:27.044016 29652 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:27.045239 29652 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.054216 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:27.046188 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:27.077631 29281 leader_election.cc:341] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:27.079335 29281 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:27.087764 30076 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:06:27.108139 29341 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:27.215799 29652 ts_tablet_manager.cc:1428] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.201s user 0.011s sys 0.005s
I20260812 19:06:27.314287 29650 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:27.315152 29650 ts_tablet_manager.cc:1397] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.746s user 0.011s sys 0.020s
I20260812 19:06:27.343044 29649 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:27.343796 29649 ts_tablet_manager.cc:1397] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 1.078s user 0.029s sys 0.031s
I20260812 19:06:27.323709 29577 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:27.352263 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:27.353796 30066 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:27.354741 30066 raft_consensus.cc:686] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:27.356333 30066 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.357445 29649 raft_consensus.cc:348] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.359615 29649 raft_consensus.cc:374] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:27.359989 29649 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:27.360843 29649 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.366771 29649 ts_tablet_manager.cc:1428] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.022s user 0.011s sys 0.001s
I20260812 19:06:27.368611 29649 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:27.379192 29650 raft_consensus.cc:348] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.381397 29650 raft_consensus.cc:374] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:27.382118 29650 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:27.383586 29650 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.391079 29650 ts_tablet_manager.cc:1428] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.075s user 0.007s sys 0.006s
I20260812 19:06:27.392922 29650 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:27.396333 29649 tablet_bootstrap.cc:654] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:27.436592 29649 tablet_bootstrap.cc:492] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:27.437389 29649 ts_tablet_manager.cc:1397] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.069s user 0.019s sys 0.008s
W20260812 19:06:27.438109 29399 consensus_peers.cc:487] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Couldn't send request to peer dd7e8c7a8bc54e73a39411cfc241fe9c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:27.449896 29649 raft_consensus.cc:348] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.451095 29649 raft_consensus.cc:374] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:27.451679 29649 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:27.452800 29649 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.455060 29650 tablet_bootstrap.cc:654] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:27.456779 29649 ts_tablet_manager.cc:1428] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.019s user 0.010s sys 0.001s
I20260812 19:06:27.459265 29649 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:27.485761 29649 tablet_bootstrap.cc:654] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:27.531646 29650 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:27.892524 29650 ts_tablet_manager.cc:1397] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.500s user 0.029s sys 0.016s
I20260812 19:06:27.821468 29982 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:27.833280 30085 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:27.894189 29982 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.894405 30085 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:27.847175 30103 log.cc:924] Time spent T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Append to log took a long time: real 0.433s user 0.003s sys 0.005s
I20260812 19:06:27.888190 29906 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:27.898532 29880 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:27.900246 29880 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:27.902740 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:27.906975 29906 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.907998 29880 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:27.912499 29650 raft_consensus.cc:348] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.914311 29906 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:27.913682 29650 raft_consensus.cc:374] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:27.915796 29650 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:27.916741 29650 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.922613 29982 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:27.919358 29650 ts_tablet_manager.cc:1428] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.026s user 0.007s sys 0.003s
I20260812 19:06:27.926694 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:27.930794 29650 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:27.922801 30085 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:27.920521 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:27.931133 29528 leader_election.cc:341] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:27.939168 29452 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:27.965359 29650 tablet_bootstrap.cc:654] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:27.992164 29650 tablet_bootstrap.cc:492] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:27.992816 29650 ts_tablet_manager.cc:1397] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.062s user 0.021s sys 0.018s
I20260812 19:06:27.996996 29179 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 54) took 1064 ms. Trace:
I20260812 19:06:27.998795 29650 raft_consensus.cc:348] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.000345 29650 raft_consensus.cc:374] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.000963 29650 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.001076 30085 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:28.002242 29650 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.002792 30085 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.000516 29649 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:28.005580 29649 ts_tablet_manager.cc:1397] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.547s user 0.032s sys 0.096s
I20260812 19:06:28.004669 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:27.999159 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.014459 29577 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:28.022349 29649 raft_consensus.cc:348] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.024235 29649 raft_consensus.cc:374] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.023167 30085 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.024736 29649 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.027223 29650 ts_tablet_manager.cc:1428] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.034s user 0.009s sys 0.007s
I20260812 19:06:28.026532 29649 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:27.997736 29179 rpcz_store.cc:276] 0812 19:06:26.938155 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:26.938633 (+ 478us) service_pool.cc:227] Handling call
0812 19:06:27.996782 (+1058149us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:06:27.335835 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:06:27.369492 (+ 33657us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:06:27.369568 (+ 76us) write_op.cc:435] Acquired schema lock
0812 19:06:27.369613 (+ 45us) tablet.cc:660] Decoding operations
0812 19:06:27.374570 (+ 4957us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:06:27.374681 (+ 111us) write_op.cc:641] Partition lock acquired for write op
0812 19:06:27.374725 (+ 44us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:06:27.375108 (+ 383us) tablet.cc:699] Row locks acquired
0812 19:06:27.375137 (+ 29us) write_op.cc:260] PREPARE: finished
0812 19:06:27.375395 (+ 258us) write_op.cc:270] Start()
0812 19:06:27.375664 (+ 269us) write_op.cc:276] Timestamp: P: 1786561587375304 usec, L: 0
0812 19:06:27.375717 (+ 53us) op_driver.cc:331] REPLICATION: starting
0812 19:06:27.379246 (+ 3529us) log.cc:842] Serialized 540 byte log entry
0812 19:06:27.846679 (+467433us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b2000170478. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69aa3f11 00007f9c69ac5b09 00007f9c69a90c43 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499 00007f9c69a99461 00007f9c69a9875d 00007f9c7014a204
0812 19:06:27.964137 (+117458us) op_driver.cc:444] REPLICATION: finished
0812 19:06:27.969995 (+ 5858us) write_op.cc:301] APPLY: starting
0812 19:06:27.970211 (+ 216us) tablet.cc:1365] starting BulkCheckPresence
0812 19:06:27.970505 (+ 294us) tablet.cc:1368] finished BulkCheckPresence
0812 19:06:27.970581 (+ 76us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:06:27.971180 (+ 599us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:06:27.971784 (+ 604us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:06:27.971977 (+ 193us) write_op.cc:312] APPLY: finished
0812 19:06:27.972527 (+ 550us) log.cc:842] Serialized 22 byte log entry
0812 19:06:27.979752 (+ 7225us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:06:27.979990 (+ 238us) write_op.cc:454] Released schema lock
0812 19:06:27.980146 (+ 156us) write_op.cc:341] FINISH: Updating metrics
0812 19:06:27.980910 (+ 764us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:06:27.981016 (+ 106us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":5046,"apply.run_cpu_time_us":4884,"apply.run_wall_time_us":11823,"num_ops":1,"prepare.queue_time_us":282438,"prepare.run_cpu_time_us":32769,"prepare.run_wall_time_us":88808,"raft.queue_time_us":63295,"raft.run_cpu_time_us":4523,"raft.run_wall_time_us":5124,"replication_time_us":588171,"spinlock_wait_cycles":128738176,"thread_start_us":125128,"threads_started":4,"wal-append.queue_time_us":34251}]]}
I20260812 19:06:28.032130 29649 ts_tablet_manager.cc:1428] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.026s user 0.013s sys 0.002s
I20260812 19:06:28.037861 29650 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:28.037866 29649 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:28.045012 29652 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:28.047115 29399 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:28.024880 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.048638 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55520 (request call id 67) took 1020 ms. Trace:
I20260812 19:06:28.049389 29176 rpcz_store.cc:276] 0812 19:06:27.027987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:27.028242 (+ 255us) service_pool.cc:227] Handling call
0812 19:06:28.048479 (+1020237us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260812 19:06:28.051941 30081 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:28.052665 30081 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:28.056772 29650 tablet_bootstrap.cc:654] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:28.060359 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 847064us
I20260812 19:06:28.065469 30081 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.095070 30085 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.065518 29880 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.035s
I20260812 19:06:28.105563 30032 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.111065 30085 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.112212 30032 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.123355 30032 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.123943 30120 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:28.126446 30120 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.128360 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.131263 29906 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.125183 30085 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.141891 30032 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:28.132084 29906 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.192859 29650 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:28.206576 29650 ts_tablet_manager.cc:1397] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.169s user 0.035s sys 0.028s
I20260812 19:06:28.208799 29580 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.195501 30120 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.193218 30036 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:28.143167 30032 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.212483 29649 tablet_bootstrap.cc:654] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.212661 29906 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.213825 30036 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.189568 30037 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.237635 30037 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.239145 29650 raft_consensus.cc:348] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.242908 29650 raft_consensus.cc:374] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.243170 29650 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.246611 29650 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.249645 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.250872 29652 tablet_bootstrap.cc:654] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:28.254460 29528 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.223165 30120 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:28.258617 30036 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.259516 30077 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:28.260346 30077 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.262921 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.264053 29572 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:28.241408 29578 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.264902 30081 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.274804 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.278851 29281 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:28.278030 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.267069 29906 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:28.275132 30037 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.280750 29906 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.284129 30077 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.284904 30081 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.285523 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.285579 29650 ts_tablet_manager.cc:1428] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.079s user 0.005s sys 0.009s
I20260812 19:06:28.287730 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:28.288316 29575 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.289078 29650 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:28.291123 29906 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.293514 30127 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:28.293926 30127 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:28.292760 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.294322 30127 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.298466 29880 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.294322 29178 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:06:28.270289 30120 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.300096 29880 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.305703 30032 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.314867 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.316239 29339 raft_consensus.cc:2379] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:28.305649 30130 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260812 19:06:28.304986 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.326738 29982 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:28.329141 30032 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:28.332159 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.332454 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.332875 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.339893 29880 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.320922 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.343477 30032 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.345498 29332 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.349148 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.340423 29982 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.353513 29880 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.364346 29982 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.368263 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.368803 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.369518 29575 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:28.365705 29649 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:28.370584 29649 ts_tablet_manager.cc:1397] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.333s user 0.017s sys 0.020s
I20260812 19:06:28.366680 30032 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.366192 29880 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.370338 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.372223 29650 tablet_bootstrap.cc:654] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.359011 30120 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.374379 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:28.378690 29399 leader_election.cc:341] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.385653 29399 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:28.378878 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.389099 30074 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:28.385210 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.389979 30074 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:28.391633 30074 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.400965 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:28.406991 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.403016 29880 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.413477 29649 raft_consensus.cc:348] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.414721 29649 raft_consensus.cc:374] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.415007 29649 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.415983 29649 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.412665 29453 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.430491 29652 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:28.430397 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.431116 29652 ts_tablet_manager.cc:1397] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.387s user 0.039s sys 0.041s
I20260812 19:06:28.437472 29649 ts_tablet_manager.cc:1428] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.067s user 0.003s sys 0.005s
I20260812 19:06:28.439474 29652 raft_consensus.cc:348] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.443471 29652 raft_consensus.cc:374] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.443869 29652 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.466135 29652 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.472520 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.449117 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.436561 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.476137 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.450397 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.477010 29456 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:28.473719 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.487680 29649 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
W20260812 19:06:28.488286 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 71162.1us
I20260812 19:06:28.490337 29652 ts_tablet_manager.cc:1428] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.059s user 0.006s sys 0.001s
I20260812 19:06:28.477684 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.477689 29953 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260812 19:06:28.477862 30081 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260812 19:06:28.501528 29652 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
W20260812 19:06:28.536397 29403 leader_election.cc:341] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:28.542869 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08001165c0 after lost signal to thread 27450
W20260812 19:06:28.543987 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08000c2c40 after lost signal to thread 29236
W20260812 19:06:28.547119 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08002141a0 after lost signal to thread 29379
W20260812 19:06:28.548125 29374 debug-util.cc:398] Leaking SignalData structure 0x7b0800244be0 after lost signal to thread 29496
W20260812 19:06:28.549053 29374 debug-util.cc:398] Leaking SignalData structure 0x7b0800236640 after lost signal to thread 29622
I20260812 19:06:28.549454 30037 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:28.548975 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.548898 29282 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.551081 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.552348 29282 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.557875 30037 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:06:28.559115 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 56201.1us
I20260812 19:06:28.559921 29649 tablet_bootstrap.cc:654] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.562801 29530 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.537113 30127 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.569751 30159 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.578416 30129 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.560932 29403 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.581202 29341 raft_consensus.cc:2379] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:28.599059 30114 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:28.599741 30114 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.577891 30159 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:28.601998 30159 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.605012 29282 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.585875 29179 catalog_manager.cc:5495] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:28.606139 29282 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.609287 29282 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.610292 29982 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.608952 30036 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.615198 30036 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:06:28.616667 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 65309.9us
I20260812 19:06:28.622027 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.610615 30077 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.626623 29652 tablet_bootstrap.cc:654] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.623777 30114 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.630204 30159 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:28.624344 29403 leader_election.cc:341] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.641403 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.644428 30121 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.648036 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:28.648780 29459 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.650499 30127 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.651489 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.656812 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.664090 29282 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:06:28.666092 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 45421.9us
I20260812 19:06:28.668008 30036 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.669683 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:28.670610 29403 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.671123 29577 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.675992 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.676523 30074 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:06:28.680078 29403 leader_election.cc:341] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.680830 29403 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.691100 29650 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:28.691632 29650 ts_tablet_manager.cc:1397] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.403s user 0.018s sys 0.015s
I20260812 19:06:28.686635 30074 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.684228 29459 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:28.708917 29282 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:28.701398 30112 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.710738 30130 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:28.711426 30130 raft_consensus.cc:686] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:28.712288 30112 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.713016 30130 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.719342 30112 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.721104 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.722155 29450 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:28.723820 29528 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:28.707944 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.726079 29982 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.725085 30112 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:28.731220 30112 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:28.731807 30112 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.737828 29650 raft_consensus.cc:348] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.739089 29650 raft_consensus.cc:374] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.739411 29650 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.740181 29650 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.747975 29650 ts_tablet_manager.cc:1428] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.056s user 0.006s sys 0.002s
W20260812 19:06:28.748980 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 183868us
I20260812 19:06:28.751009 29530 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.751986 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:06:28.752687 29530 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.752966 29530 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.753691 29530 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.754828 29530 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.756776 29650 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:28.758224 30114 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.754416 30159 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.767678 29652 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:28.768200 29652 ts_tablet_manager.cc:1397] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.270s user 0.024s sys 0.008s
I20260812 19:06:28.768724 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.773775 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.754225 30166 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.754542 29906 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.779307 29530 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:06:28.780586 29530 leader_election.cc:341] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.782015 30114 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.786466 29649 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:28.788188 29649 ts_tablet_manager.cc:1397] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.313s user 0.008s sys 0.026s
I20260812 19:06:28.784060 29577 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:28.789464 30085 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.776719 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:28.800648 29652 raft_consensus.cc:348] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.802882 29652 raft_consensus.cc:374] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.803260 29652 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.803954 29652 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.814831 29649 raft_consensus.cc:348] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.818152 29649 raft_consensus.cc:374] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.818439 29649 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.806046 29530 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.818917 29649 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.830288 30114 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:28.831561 29649 ts_tablet_manager.cc:1428] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.043s user 0.003s sys 0.004s
I20260812 19:06:28.832842 29649 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:28.838779 29652 ts_tablet_manager.cc:1428] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.070s user 0.006s sys 0.002s
I20260812 19:06:28.850456 29652 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:28.861091 29650 tablet_bootstrap.cc:654] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.866356 30130 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.868286 30130 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.870719 30130 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.873198 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.874751 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:28.875666 29575 raft_consensus.cc:2379] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:28.876230 29652 tablet_bootstrap.cc:654] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.876555 29649 tablet_bootstrap.cc:654] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:28.880151 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:28.881346 30130 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.887363 30112 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.901455 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:28.902891 29530 leader_election.cc:341] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.902621 30114 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.904896 30114 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.895560 29178 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:28.907261 30114 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.908529 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.909315 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:28.910219 29528 leader_election.cc:341] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:28.911212 29530 leader_election.cc:341] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.911639 29530 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:28.912958 30159 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.922174 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:28.922955 29446 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.912914 30112 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:28.934878 30127 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:28.936555 30127 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.941021 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:28.942049 29453 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:28.943853 29650 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:28.944317 29650 ts_tablet_manager.cc:1397] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.188s user 0.017s sys 0.010s
I20260812 19:06:28.943881 29282 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:28.945099 30077 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:28.945663 30077 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:28.946889 30077 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:28.959591 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:28.964668 29281 leader_election.cc:341] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:28.957769 30127 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:28.957656 29650 raft_consensus.cc:348] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.968953 29650 raft_consensus.cc:374] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:28.969193 29650 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:28.973317 29650 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:28.958550 30130 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:28.977023 29176 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:28.986258 29649 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:28.986660 29649 ts_tablet_manager.cc:1397] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.154s user 0.007s sys 0.031s
I20260812 19:06:28.991063 29650 ts_tablet_manager.cc:1428] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.046s user 0.006s sys 0.005s
I20260812 19:06:28.992177 29650 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:29.008344 29652 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:29.008249 29649 raft_consensus.cc:348] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.009028 29649 raft_consensus.cc:374] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.009291 29649 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.009882 29649 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.012337 29649 ts_tablet_manager.cc:1428] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.025s user 0.003s sys 0.006s
I20260812 19:06:29.013221 29649 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:29.015272 29446 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:29.016887 29528 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.018844 30112 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:29.008800 29652 ts_tablet_manager.cc:1397] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.159s user 0.018s sys 0.009s
I20260812 19:06:29.019869 30112 raft_consensus.cc:686] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:29.024140 29652 raft_consensus.cc:348] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.023690 30112 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.024935 29652 raft_consensus.cc:374] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.026988 29652 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.027627 29652 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.029793 29652 ts_tablet_manager.cc:1428] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.009s user 0.001s sys 0.005s
I20260812 19:06:29.032033 29649 tablet_bootstrap.cc:654] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:29.035269 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:29.038765 30077 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.051718 30077 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.045982 29652 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:29.054767 29880 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.055843 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:29.065012 29880 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.067114 29880 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.070194 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:29.070010 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:29.071120 29577 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
W20260812 19:06:29.072006 29403 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.063836 29650 tablet_bootstrap.cc:654] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.065194 29451 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.056182 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:06:29.083455 29281 leader_election.cc:341] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.122857 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.124442 29649 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:29.124926 29649 ts_tablet_manager.cc:1397] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.112s user 0.012s sys 0.015s
I20260812 19:06:29.125013 30081 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:29.125650 30081 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:29.084909 29880 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260812 19:06:29.126227 30081 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.129556 29649 raft_consensus.cc:348] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.130424 29649 raft_consensus.cc:374] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.130741 29649 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.131552 29649 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.134275 29953 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:29.135861 29649 ts_tablet_manager.cc:1428] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260812 19:06:29.136988 29649 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:29.152416 29451 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:29.153856 29282 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:29.103791 30130 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260812 19:06:29.154840 30127 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:29.178946 30127 raft_consensus.cc:686] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:29.179911 30127 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.206946 29652 tablet_bootstrap.cc:654] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.213841 29176 catalog_manager.cc:5495] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:29.218137 30130 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:29.234581 29649 tablet_bootstrap.cc:654] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.220206 30081 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.263540 29650 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:29.264137 29650 ts_tablet_manager.cc:1397] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.272s user 0.008s sys 0.016s
I20260812 19:06:29.266163 30159 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.277313 30159 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.280108 30159 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:29.281140 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:29.286845 29528 leader_election.cc:341] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.291009 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:29.301091 30177 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.302083 30077 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.301714 30177 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.302532 30077 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:29.306830 29530 leader_election.cc:341] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.308571 29530 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.309882 29650 raft_consensus.cc:348] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.314117 29650 raft_consensus.cc:374] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.314337 29650 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.314805 29650 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.306465 30077 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.312186 30112 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:29.323498 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:29.324813 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:29.325632 29450 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:29.327003 30081 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.320832 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:29.328622 29282 leader_election.cc:341] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.326519 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:29.321193 30177 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:29.334807 29281 leader_election.cc:341] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.335243 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:29.335881 29572 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.338347 29528 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.339916 30177 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.338379 29650 ts_tablet_manager.cc:1428] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.074s user 0.007s sys 0.002s
I20260812 19:06:29.335300 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:29.344293 29650 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:29.349660 30129 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:29.374459 29176 catalog_manager.cc:5495] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:29.382939 29650 tablet_bootstrap.cc:654] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.415374 29649 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:29.415952 29649 ts_tablet_manager.cc:1397] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.279s user 0.015s sys 0.012s
I20260812 19:06:29.419839 30177 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:29.417579 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:29.436184 29649 raft_consensus.cc:348] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.443262 29649 raft_consensus.cc:374] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.443619 29649 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.444465 29649 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:29.467299 29403 leader_election.cc:341] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.451315 29572 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
W20260812 19:06:29.479979 29530 consensus_peers.cc:487] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:29.486011 29652 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:29.488310 29652 ts_tablet_manager.cc:1397] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.443s user 0.023s sys 0.040s
I20260812 19:06:29.490866 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.495811 30074 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:29.500255 30074 raft_consensus.cc:686] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:29.502202 30074 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.522871 29649 ts_tablet_manager.cc:1428] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.107s user 0.033s sys 0.031s
I20260812 19:06:29.523929 29649 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:29.530311 29650 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:29.530964 29650 ts_tablet_manager.cc:1397] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.187s user 0.008s sys 0.019s
I20260812 19:06:29.533206 29649 tablet_bootstrap.cc:654] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.536146 29650 raft_consensus.cc:348] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.536711 29650 raft_consensus.cc:374] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.536922 29650 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.537431 29650 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.540453 29652 raft_consensus.cc:348] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.541146 29652 raft_consensus.cc:374] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.541306 30219 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.541502 29176 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:29.543442 29650 ts_tablet_manager.cc:1428] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260812 19:06:29.542419 30219 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.544420 29650 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:29.541424 29652 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.546826 29652 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.549973 29652 ts_tablet_manager.cc:1428] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.061s user 0.004s sys 0.002s
I20260812 19:06:29.554201 29652 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:29.558211 30159 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:29.575368 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:29.580862 29530 leader_election.cc:341] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.584841 30219 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:29.588246 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:29.590459 29451 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:29.592077 29528 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.596508 30159 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:29.597389 30159 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:29.598063 30159 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.601078 29649 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:29.601569 29649 ts_tablet_manager.cc:1397] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.078s user 0.011s sys 0.010s
I20260812 19:06:29.603607 29652 tablet_bootstrap.cc:654] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.606554 29649 raft_consensus.cc:348] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.610685 29649 raft_consensus.cc:374] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.611019 29649 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.611933 29649 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.616561 29649 ts_tablet_manager.cc:1428] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260812 19:06:29.627388 29650 tablet_bootstrap.cc:654] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.628672 29649 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:29.639941 30159 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.643843 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:29.646931 29530 leader_election.cc:341] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.658136 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:29.659046 29450 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.663383 29652 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:29.654421 30159 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:29.731668 29652 ts_tablet_manager.cc:1397] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.178s user 0.007s sys 0.021s
W20260812 19:06:29.734578 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 66865.9us
W20260812 19:06:29.735481 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 67538.2us
I20260812 19:06:29.735759 29650 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:29.736172 29650 ts_tablet_manager.cc:1397] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.192s user 0.031s sys 0.019s
I20260812 19:06:29.748699 29450 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:29.750183 29528 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.752138 30159 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:29.752612 30159 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:29.753783 30159 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.756634 29652 raft_consensus.cc:348] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.757369 29652 raft_consensus.cc:374] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.757622 29652 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.758232 29652 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.761621 29652 ts_tablet_manager.cc:1428] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.029s user 0.004s sys 0.001s
I20260812 19:06:29.762614 29652 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:29.780644 29649 tablet_bootstrap.cc:654] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.803531 29650 raft_consensus.cc:348] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.803225 29179 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:29.810153 29650 raft_consensus.cc:374] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.811679 29650 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.812312 29650 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.817600 29650 ts_tablet_manager.cc:1428] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.081s user 0.008s sys 0.000s
I20260812 19:06:29.823666 29650 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:29.826087 30127 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:29.831308 29652 tablet_bootstrap.cc:654] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.842906 30129 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:29.847504 29650 tablet_bootstrap.cc:654] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.849710 29649 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:29.850199 29649 ts_tablet_manager.cc:1397] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.222s user 0.015s sys 0.019s
I20260812 19:06:29.851871 30219 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.852743 30219 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.863657 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:29.865243 29530 leader_election.cc:341] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:29.867890 30219 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:29.866643 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:29.869853 29649 raft_consensus.cc:348] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.870637 29649 raft_consensus.cc:374] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.870904 29649 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.871742 29649 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.871212 29450 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:29.872855 29982 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:29.873437 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:29.873842 29649 ts_tablet_manager.cc:1428] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.023s user 0.004s sys 0.001s
I20260812 19:06:29.873579 29982 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.878566 30159 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:29.879717 30159 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:29.880051 30159 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.888513 30159 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.889541 29650 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:29.890507 29650 ts_tablet_manager.cc:1397] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.067s user 0.004s sys 0.019s
I20260812 19:06:29.891731 29649 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:29.897789 29652 tablet_bootstrap.cc:492] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:29.898304 29652 ts_tablet_manager.cc:1397] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.136s user 0.014s sys 0.006s
I20260812 19:06:29.903963 29982 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.906451 29652 raft_consensus.cc:348] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.907086 29652 raft_consensus.cc:374] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.905774 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:29.907362 29652 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.908568 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:29.909359 29572 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:29.907995 29652 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.918632 29650 raft_consensus.cc:348] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.919338 29650 raft_consensus.cc:374] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:29.919612 29650 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:29.920156 29650 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.922075 29339 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.923750 29332 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:29.928790 29403 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:29.930109 29953 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:29.930531 29953 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:29.930912 29953 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.933158 29650 ts_tablet_manager.cc:1428] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.042s user 0.002s sys 0.005s
I20260812 19:06:29.935222 30081 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:29.936017 30074 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:29.943807 29649 tablet_bootstrap.cc:654] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:29.946161 29650 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:29.948493 29652 ts_tablet_manager.cc:1428] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.050s user 0.009s sys 0.000s
I20260812 19:06:29.959239 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:29.959870 29342 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:29.966151 29652 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:29.978585 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:29.975440 30159 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:29.980715 29953 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:29.982174 29450 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:29.985037 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:29.992699 29953 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:29.998307 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:29.999192 29578 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:30.000200 29652 tablet_bootstrap.cc:654] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.000838 29953 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.002754 29953 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.011220 29953 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.011926 29650 tablet_bootstrap.cc:654] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.016928 30127 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.017175 29399 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:30.017480 30127 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.020010 30127 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.018240 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.022082 29332 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:30.023061 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.023720 29578 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:30.024964 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:30.019950 29342 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:30.032171 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:30.032519 30074 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:06:30.033035 30127 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.033388 30127 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.033607 29403 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:30.033685 30127 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.032868 29450 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:30.041031 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.041759 29578 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:30.044569 29530 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:30.045831 30159 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:30.046305 30159 raft_consensus.cc:686] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:30.047081 30159 consensus_queue.cc:237] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.048358 30074 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.054469 30127 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.056479 30074 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.063894 30127 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.064965 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:30.065135 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:30.065799 30074 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.065654 29578 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.070359 29650 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:30.070796 29650 ts_tablet_manager.cc:1397] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.137s user 0.015s sys 0.001s
I20260812 19:06:30.084803 29880 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.083340 29578 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:06:30.089361 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:30.085274 29880 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.093981 29880 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.093694 30032 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:30.094877 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.095532 30032 raft_consensus.cc:686] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:06:30.096720 29403 leader_election.cc:341] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.097581 29649 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:30.098107 29649 ts_tablet_manager.cc:1397] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.219s user 0.027s sys 0.028s
I20260812 19:06:30.085088 29176 catalog_manager.cc:5495] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:30.097512 30032 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:30.109720 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:30.115819 29649 raft_consensus.cc:348] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.116598 29649 raft_consensus.cc:374] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.116850 29649 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.117506 29649 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.119581 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.120671 29578 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:30.133399 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:30.134480 29880 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.134845 29880 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.135118 29880 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.140908 29650 raft_consensus.cc:348] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.150048 29650 raft_consensus.cc:374] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.150436 29650 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.151351 29650 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.149231 29880 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.154852 29880 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.157286 29649 ts_tablet_manager.cc:1428] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.059s user 0.007s sys 0.005s
I20260812 19:06:30.159585 29650 ts_tablet_manager.cc:1428] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.089s user 0.012s sys 0.001s
I20260812 19:06:30.153508 30159 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260812 19:06:30.168641 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:30.170118 29530 consensus_peers.cc:487] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260812 19:06:30.170698 29403 leader_election.cc:341] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.183741 30074 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.185997 29450 raft_consensus.cc:2379] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:30.188040 29179 catalog_manager.cc:5495] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:30.189407 29652 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:30.192281 29652 ts_tablet_manager.cc:1397] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.235s user 0.003s sys 0.015s
I20260812 19:06:30.192713 29649 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:30.198374 29650 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:30.181348 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:30.202437 29575 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.230062 29652 raft_consensus.cc:348] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.230806 29652 raft_consensus.cc:374] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.231081 29652 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.233367 29652 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.235270 29649 tablet_bootstrap.cc:654] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.234841 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:30.247539 30074 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.249040 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:30.250423 29578 raft_consensus.cc:2379] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:30.251833 29399 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:30.254225 29650 tablet_bootstrap.cc:654] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.257145 29652 ts_tablet_manager.cc:1428] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.064s user 0.004s sys 0.005s
I20260812 19:06:30.256009 30074 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260812 19:06:30.271324 29652 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:30.279402 29575 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:30.282851 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:30.283900 30074 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:30.284570 30074 raft_consensus.cc:686] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:30.285374 30074 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.297254 29652 tablet_bootstrap.cc:654] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.302366 29649 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:30.302817 29649 ts_tablet_manager.cc:1397] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.115s user 0.005s sys 0.027s
I20260812 19:06:30.305523 29649 raft_consensus.cc:348] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.306272 29649 raft_consensus.cc:374] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.306522 29649 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.310956 29650 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:30.306613 29176 catalog_manager.cc:5495] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:30.311473 29650 ts_tablet_manager.cc:1397] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.113s user 0.001s sys 0.019s
I20260812 19:06:30.325748 29649 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.325999 29650 raft_consensus.cc:348] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.326809 29650 raft_consensus.cc:374] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.327088 29650 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.327780 29650 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.329025 29649 ts_tablet_manager.cc:1428] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.026s user 0.009s sys 0.000s
I20260812 19:06:30.332381 29649 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:30.333865 29650 ts_tablet_manager.cc:1428] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.022s user 0.006s sys 0.012s
I20260812 19:06:30.335259 29650 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:30.342810 29652 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:30.343402 29652 ts_tablet_manager.cc:1397] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.072s user 0.015s sys 0.003s
I20260812 19:06:30.348670 29649 tablet_bootstrap.cc:654] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.350296 29652 raft_consensus.cc:348] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.352380 29652 raft_consensus.cc:374] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.352672 29652 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.353766 29652 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.351962 29650 tablet_bootstrap.cc:654] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.362840 29652 ts_tablet_manager.cc:1428] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.019s user 0.010s sys 0.000s
I20260812 19:06:30.380610 29652 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:30.393793 29649 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:30.394536 29649 ts_tablet_manager.cc:1397] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.062s user 0.011s sys 0.021s
I20260812 19:06:30.397703 29649 raft_consensus.cc:348] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.398500 29649 raft_consensus.cc:374] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.398833 29649 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.399644 29649 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.402098 29649 ts_tablet_manager.cc:1428] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:06:30.406231 29649 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:30.419045 29652 tablet_bootstrap.cc:654] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.435103 30114 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.435884 30114 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.440426 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.441474 29342 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:30.442293 29982 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.443351 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:30.443166 29982 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.444325 30219 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.444944 30219 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.445540 30219 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.448962 30127 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.449729 30127 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.452795 30227 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.455323 29649 tablet_bootstrap.cc:654] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.456493 30127 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.457535 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:30.453649 30227 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.464025 30227 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.473905 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:30.474973 29451 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:30.475991 30159 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.476850 29282 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:30.466213 30218 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260812 19:06:30.480572 29650 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:30.481652 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.482396 30227 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.483260 29650 ts_tablet_manager.cc:1397] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.148s user 0.044s sys 0.013s
I20260812 19:06:30.485419 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.482052 30219 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.468075 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.480388 29982 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.486166 30177 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.487541 29332 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:30.493566 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:30.487583 29578 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:30.488116 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.495004 29575 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 0.
I20260812 19:06:30.495363 30227 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.495986 30227 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.487140 30074 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.498725 29281 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:30.498585 30074 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.494338 29339 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.499799 30177 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.500164 30129 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:30.500542 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:30.503731 30177 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:30.476833 30159 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.495594 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:30.497288 29650 raft_consensus.cc:348] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.478817 30114 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:30.506064 29650 raft_consensus.cc:374] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.504642 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.491104 30219 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:30.513247 29650 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.516531 29652 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:30.517019 29652 ts_tablet_manager.cc:1397] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.137s user 0.032s sys 0.041s
I20260812 19:06:30.518208 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:30.520280 29650 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:30.521782 29528 leader_election.cc:341] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.523583 29650 ts_tablet_manager.cc:1428] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.038s user 0.001s sys 0.005s
I20260812 19:06:30.514907 30081 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:30.524662 29650 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:30.524519 30081 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:30.525312 30081 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:30.526752 29530 leader_election.cc:341] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.527172 29530 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:30.517705 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:30.531234 29453 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:30.532168 29652 raft_consensus.cc:348] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.535519 30159 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:30.535889 30219 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:30.538882 30227 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.553889 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.555698 29652 raft_consensus.cc:374] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.567286 29652 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.561352 30081 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.571352 30227 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.571246 29339 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:30.566505 29650 tablet_bootstrap.cc:654] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.578794 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:30.580365 29332 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 1: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:06:30.579988 29652 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:30.581679 29530 leader_election.cc:341] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.578999 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:30.584694 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:30.588940 29652 ts_tablet_manager.cc:1428] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.072s user 0.008s sys 0.002s
I20260812 19:06:30.584663 29575 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.586181 30219 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:30.599789 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:30.602680 30219 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:30.605553 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:30.605479 29652 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:30.605683 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:30.607813 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:30.607291 30219 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.608609 29453 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:30.609733 29880 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260812 19:06:30.611042 29399 leader_election.cc:341] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.613211 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:30.611500 30074 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:30.617165 29403 leader_election.cc:341] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.617569 29403 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:30.618971 29880 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:30.621564 30129 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:30.627180 29649 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:30.627605 29649 ts_tablet_manager.cc:1397] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.222s user 0.038s sys 0.043s
I20260812 19:06:30.589653 30081 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:30.628947 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:30.660976 29528 leader_election.cc:341] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:30.661473 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:30.667140 30159 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:30.697650 30272 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.074s user 0.000s sys 0.006s
I20260812 19:06:30.699128 30159 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:30.681893 29575 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
W20260812 19:06:30.756000 29403 consensus_peers.cc:487] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:30.767625 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:30.767308 30032 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:30.769555 30032 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:30.770720 30032 raft_consensus.cc:686] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:30.772305 30032 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.765868 29650 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:30.780483 29650 ts_tablet_manager.cc:1397] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.256s user 0.026s sys 0.049s
I20260812 19:06:30.784313 29650 raft_consensus.cc:348] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.785135 29650 raft_consensus.cc:374] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.785418 29650 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.784854 29342 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:30.792640 29652 tablet_bootstrap.cc:654] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:30.797056 29650 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.799484 29650 ts_tablet_manager.cc:1428] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.019s user 0.004s sys 0.004s
I20260812 19:06:30.799607 29649 raft_consensus.cc:348] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.800556 29649 raft_consensus.cc:374] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:30.800869 29649 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:30.801517 29649 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:30.804996 29649 ts_tablet_manager.cc:1428] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.177s user 0.008s sys 0.000s
I20260812 19:06:30.806205 29649 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:30.800537 29650 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:30.816536 29176 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
W20260812 19:06:30.819679 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 158747us
I20260812 19:06:30.864153 30230 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:30.953363 30230 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.023351 30230 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:31.023725 29649 tablet_bootstrap.cc:654] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:31.039218 29650 tablet_bootstrap.cc:654] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:31.067740 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 182116us
W20260812 19:06:31.076750 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 145657us
I20260812 19:06:31.086118 30278 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:31.086711 30278 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.088649 30278 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:31.089150 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:31.091223 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:31.097854 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 89637.5us
I20260812 19:06:31.108387 29880 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:31.131989 29649 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:31.137768 29649 ts_tablet_manager.cc:1397] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.332s user 0.012s sys 0.007s
I20260812 19:06:31.152392 29649 raft_consensus.cc:348] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.153411 29649 raft_consensus.cc:374] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.153759 29649 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
W20260812 19:06:31.154402 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 224083us
W20260812 19:06:31.154699 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:31.154556 29649 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.162321 29652 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:31.162839 29652 ts_tablet_manager.cc:1397] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.558s user 0.012s sys 0.029s
I20260812 19:06:31.168558 29649 ts_tablet_manager.cc:1428] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.030s user 0.006s sys 0.006s
I20260812 19:06:31.193727 29459 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:31.197886 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 68351.9us
W20260812 19:06:31.198583 29282 leader_election.cc:341] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.205137 29649 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
W20260812 19:06:31.230665 29281 leader_election.cc:341] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.231127 29281 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:31.233117 30284 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:31.265654 30127 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:31.301548 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 69700us
I20260812 19:06:31.263286 30177 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260812 19:06:31.287475 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:31.277531 30284 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.369536 29650 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:31.370319 29650 ts_tablet_manager.cc:1397] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.570s user 0.021s sys 0.021s
I20260812 19:06:31.337076 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:31.376020 29652 raft_consensus.cc:348] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.377202 29652 raft_consensus.cc:374] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.377490 29652 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:31.378125 29652 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:31.396588 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 104879us
I20260812 19:06:31.397653 29652 ts_tablet_manager.cc:1428] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.235s user 0.006s sys 0.011s
I20260812 19:06:31.398326 30177 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:31.401047 30177 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:31.401993 29399 leader_election.cc:341] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.408572 30284 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:31.422590 30322 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260812 19:06:31.440544 29650 raft_consensus.cc:348] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.441555 29650 raft_consensus.cc:374] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.443572 29652 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:31.441859 29650 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:31.459695 29650 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.489329 29650 ts_tablet_manager.cc:1428] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.119s user 0.007s sys 0.003s
I20260812 19:06:31.458582 30177 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:31.504099 29649 tablet_bootstrap.cc:654] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:31.507962 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:31.515324 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:31.536605 29528 leader_election.cc:341] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.544275 29650 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
W20260812 19:06:31.546872 29403 leader_election.cc:341] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.551360 29403 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:31.556918 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:31.559266 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:31.560192 29452 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:31.567188 30278 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:31.573838 29176 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:31.580615 29528 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:31.592324 29530 leader_election.cc:341] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.592662 29530 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:31.595217 30114 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:31.593673 30218 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:31.610622 30218 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:31.611282 30218 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:31.591806 30159 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260812 19:06:31.609133 29652 tablet_bootstrap.cc:654] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:31.638029 30036 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:31.642972 29334 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:31.689646 29530 leader_election.cc:341] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.693094 30074 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:31.693619 30074 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.694357 30218 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.695780 30074 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:31.698535 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:31.699987 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:31.700562 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:31.701850 29399 leader_election.cc:341] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:31.702111 29530 leader_election.cc:341] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:31.702936 29403 leader_election.cc:341] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:31.703331 29403 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:31.704366 30074 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:31.727759 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:31.728955 29453 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:31.734958 29650 tablet_bootstrap.cc:654] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:31.725733 30218 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:31.775527 29453 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:31.805876 29528 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:31.808678 30177 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:31.809525 30177 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:31.810842 30177 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.866460 29649 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:31.866972 29649 ts_tablet_manager.cc:1397] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.664s user 0.016s sys 0.041s
I20260812 19:06:31.889130 29649 raft_consensus.cc:348] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.899128 29649 raft_consensus.cc:374] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.899542 29649 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:31.912333 29649 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.925593 29649 ts_tablet_manager.cc:1428] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.058s user 0.005s sys 0.003s
I20260812 19:06:31.926693 29649 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:31.936139 29652 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:31.936606 29652 ts_tablet_manager.cc:1397] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.493s user 0.023s sys 0.030s
I20260812 19:06:31.944280 29650 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:31.944790 29650 ts_tablet_manager.cc:1397] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.401s user 0.015s sys 0.049s
I20260812 19:06:31.946499 30335 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:31.953215 30335 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.964982 29650 raft_consensus.cc:348] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.970329 30335 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:31.970994 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:31.971323 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:31.973299 29282 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:31.974946 30330 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:31.965831 29650 raft_consensus.cc:374] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.980337 29650 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:31.981881 29650 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.979635 29652 raft_consensus.cc:348] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.983175 29652 raft_consensus.cc:374] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:31.983507 29652 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:31.982805 29649 tablet_bootstrap.cc:654] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:31.984287 29652 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:31.986971 29650 ts_tablet_manager.cc:1428] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.042s user 0.007s sys 0.002s
I20260812 19:06:31.988247 29650 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:31.992597 29652 ts_tablet_manager.cc:1428] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.056s user 0.003s sys 0.004s
I20260812 19:06:31.993752 29652 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:31.999392 29649 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:31.999876 29649 ts_tablet_manager.cc:1397] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.073s user 0.010s sys 0.010s
I20260812 19:06:32.008963 29650 tablet_bootstrap.cc:654] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.010787 29649 raft_consensus.cc:348] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.017325 29649 raft_consensus.cc:374] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.018814 29649 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.019636 29649 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.015424 29652 tablet_bootstrap.cc:654] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.022759 29649 ts_tablet_manager.cc:1428] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260812 19:06:32.032213 29649 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.061233 29649 tablet_bootstrap.cc:654] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.067716 30301 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.068684 30301 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.077308 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:32.079226 29403 leader_election.cc:341] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.080268 30335 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.082448 30301 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.086095 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:32.088069 29399 leader_election.cc:341] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.088461 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:32.081300 30335 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.096786 29649 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:32.097304 29649 ts_tablet_manager.cc:1397] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.015s
I20260812 19:06:32.097131 30335 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.098860 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:32.100234 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.100749 29880 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:32.104370 29281 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:32.104044 30276 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.106302 30335 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:32.108824 30276 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.113283 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.113935 29334 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.114615 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.113926 30276 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.116413 29530 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:32.117035 29528 leader_election.cc:341] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.117219 30276 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.117861 30276 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.118310 30276 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.148649 29649 raft_consensus.cc:348] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.149535 29649 raft_consensus.cc:374] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.149797 29649 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.134526 29178 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:32.153023 29649 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.155123 29650 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:32.155627 29650 ts_tablet_manager.cc:1397] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.168s user 0.014s sys 0.041s
I20260812 19:06:32.156065 30276 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.159096 30276 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.159732 29650 raft_consensus.cc:348] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.159982 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:32.160506 29650 raft_consensus.cc:374] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.160778 29650 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
W20260812 19:06:32.161670 29528 leader_election.cc:341] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.161401 29650 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.163951 29649 ts_tablet_manager.cc:1428] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.066s user 0.005s sys 0.005s
I20260812 19:06:32.164546 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:32.165017 29649 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.165335 29334 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.166230 29650 ts_tablet_manager.cc:1428] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260812 19:06:32.166888 29652 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.167447 29652 ts_tablet_manager.cc:1397] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.174s user 0.038s sys 0.040s
I20260812 19:06:32.168534 29650 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:32.173825 30177 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.174643 30177 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.181017 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.182519 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:32.183441 29528 leader_election.cc:341] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:32.184787 29530 leader_election.cc:341] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.185171 29530 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.186760 29652 raft_consensus.cc:348] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.187436 30283 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:32.195034 29652 raft_consensus.cc:374] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.195427 29652 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.196578 29652 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.195050 29326 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.203017 30177 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.192281 29650 tablet_bootstrap.cc:654] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.212971 29652 ts_tablet_manager.cc:1428] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.045s user 0.008s sys 0.004s
I20260812 19:06:32.218838 29652 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:32.262490 29649 tablet_bootstrap.cc:654] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.256330 29334 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.293308 29652 tablet_bootstrap.cc:654] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:32.325995 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:32.335429 30177 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.336354 30177 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.348097 30114 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:32.353173 30177 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.370467 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.373273 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.382467 29650 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:32.382987 29650 ts_tablet_manager.cc:1397] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.215s user 0.016s sys 0.007s
I20260812 19:06:32.387176 29530 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
W20260812 19:06:32.389020 29530 leader_election.cc:341] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.390350 30114 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.391312 30114 raft_consensus.cc:686] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:32.392110 30114 consensus_queue.cc:237] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.392899 29650 raft_consensus.cc:348] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.393656 29650 raft_consensus.cc:374] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.382441 29459 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.393900 29650 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
W20260812 19:06:32.394963 29530 consensus_peers.cc:487] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:32.395298 29650 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.399919 29650 ts_tablet_manager.cc:1428] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260812 19:06:32.401049 29650 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:32.402140 29528 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.403368 30218 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.403755 30218 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.404114 30218 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.411615 29652 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.412220 29652 ts_tablet_manager.cc:1397] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.194s user 0.010s sys 0.013s
I20260812 19:06:32.421275 29880 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.421691 29880 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.424142 29880 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.420892 30177 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.429414 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.428185 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:32.431552 29403 leader_election.cc:341] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.430227 30177 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.436422 29575 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:32.436918 29652 raft_consensus.cc:348] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.438226 29652 raft_consensus.cc:374] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.438575 29652 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.440016 29652 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.440495 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.441426 30278 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.441774 30278 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.443307 29650 tablet_bootstrap.cc:654] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.442989 30278 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.446970 29649 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:32.447479 29649 ts_tablet_manager.cc:1397] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.283s user 0.046s sys 0.020s
I20260812 19:06:32.449056 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.450304 29652 ts_tablet_manager.cc:1428] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.038s user 0.007s sys 0.002s
I20260812 19:06:32.451190 29652 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:32.451244 29649 raft_consensus.cc:348] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.452067 29649 raft_consensus.cc:374] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.452353 29649 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.453017 29649 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.458884 29649 ts_tablet_manager.cc:1428] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
W20260812 19:06:32.461025 29530 leader_election.cc:341] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.462122 30218 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.464670 30218 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.465240 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:32.465749 29649 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.465850 29459 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.466730 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:32.470036 29530 leader_election.cc:341] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.474757 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.472474 30177 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.483928 29652 tablet_bootstrap.cc:654] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.483275 29179 catalog_manager.cc:5495] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:32.485757 30278 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.486987 29456 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:32.488624 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.489668 30177 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.494963 29649 tablet_bootstrap.cc:654] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.483183 29459 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.499902 29528 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.502704 30159 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.507364 30159 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:32.508189 30159 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.518404 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:32.519536 30278 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.519232 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:32.520900 29403 leader_election.cc:341] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.524250 29650 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:32.524775 29650 ts_tablet_manager.cc:1397] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.124s user 0.007s sys 0.019s
I20260812 19:06:32.526868 30177 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:32.527608 29650 raft_consensus.cc:348] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.528347 29650 raft_consensus.cc:374] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.528594 29650 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.529228 29650 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.542263 29652 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.542775 29652 ts_tablet_manager.cc:1397] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.092s user 0.018s sys 0.016s
I20260812 19:06:32.544734 29650 ts_tablet_manager.cc:1428] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260812 19:06:32.555208 30177 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.557082 30177 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.536816 29575 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:32.563218 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:32.564528 30328 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.565192 30328 raft_consensus.cc:686] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:32.566725 30328 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.567368 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.557554 29650 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:32.568562 29326 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.573531 29649 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:32.574331 29530 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.575026 29649 ts_tablet_manager.cc:1397] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.109s user 0.014s sys 0.024s
I20260812 19:06:32.575693 30177 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.575882 29652 raft_consensus.cc:348] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.577451 29652 raft_consensus.cc:374] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.577813 29652 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.578495 29652 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.577200 30177 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.581200 30177 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.581597 30177 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.584508 29179 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:32.586649 29652 ts_tablet_manager.cc:1428] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.043s user 0.006s sys 0.002s
I20260812 19:06:32.576433 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.587983 29459 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.588644 29652 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:32.593289 30218 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.593776 30218 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.580869 29650 tablet_bootstrap.cc:654] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.601389 30177 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.602008 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.604811 29331 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.605765 29880 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.606576 29880 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.596463 30218 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.612021 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.614982 29880 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.617416 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:32.620971 29577 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:32.618563 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.622416 29649 raft_consensus.cc:348] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.622776 30276 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.623184 29649 raft_consensus.cc:374] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.623484 29649 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.623947 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.623348 30276 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.625236 30276 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.634619 29652 tablet_bootstrap.cc:654] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.624145 29649 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.642256 29649 ts_tablet_manager.cc:1428] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260812 19:06:32.643260 29649 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.650563 29649 tablet_bootstrap.cc:654] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.655367 29652 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.655889 29652 ts_tablet_manager.cc:1397] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.001s
I20260812 19:06:32.665055 29652 raft_consensus.cc:348] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.665799 29652 raft_consensus.cc:374] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.666115 29652 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.666733 29652 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.666834 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.668720 29459 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.669051 29652 ts_tablet_manager.cc:1428] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260812 19:06:32.672344 30301 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.672433 29652 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:32.672751 30301 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.673108 30301 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:32.676494 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 52729us
W20260812 19:06:32.682921 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 48306.5us
W20260812 19:06:32.683929 29403 leader_election.cc:341] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:32.696432 30301 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.698614 29652 tablet_bootstrap.cc:654] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.719339 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:32.720464 29331 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.723560 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:32.724745 29332 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.725589 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:32.726444 29456 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.728426 30301 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.731329 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:32.732385 29577 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.737676 29649 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:32.738165 29649 ts_tablet_manager.cc:1397] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.095s user 0.021s sys 0.015s
I20260812 19:06:32.740926 29650 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:32.722275 30177 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.743072 29650 ts_tablet_manager.cc:1397] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.186s user 0.038s sys 0.061s
I20260812 19:06:32.740792 29649 raft_consensus.cc:348] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.744210 29649 raft_consensus.cc:374] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.743669 29178 catalog_manager.cc:5495] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:32.744505 29649 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.745704 29649 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.747524 29649 ts_tablet_manager.cc:1428] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260812 19:06:32.747427 29456 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.747712 30276 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.749212 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.749657 29650 raft_consensus.cc:348] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.750335 29650 raft_consensus.cc:374] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.750185 30218 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.750641 29650 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.751082 30218 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:32.751289 29650 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.749526 29577 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:32.752991 29649 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.752470 30218 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.755214 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:32.756152 29326 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.759253 30276 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.766949 29332 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.772908 29650 ts_tablet_manager.cc:1428] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.029s user 0.004s sys 0.000s
I20260812 19:06:32.774273 29650 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:32.779723 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.785280 30159 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:32.789121 30278 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.786860 29649 tablet_bootstrap.cc:654] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.789872 30278 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:32.793852 29331 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:32.795426 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:32.807790 29326 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.808120 29652 tablet_bootstrap.cc:492] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.791494 30278 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.808553 29652 ts_tablet_manager.cc:1397] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.136s user 0.010s sys 0.008s
I20260812 19:06:32.809394 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.796398 29456 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.814011 29652 raft_consensus.cc:348] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.814692 29652 raft_consensus.cc:374] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.814939 29652 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.815608 29652 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.818341 29652 ts_tablet_manager.cc:1428] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260812 19:06:32.811414 30331 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.819306 30331 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.822845 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.818183 30276 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:32.825110 29650 tablet_bootstrap.cc:654] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.825383 30276 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:32.826079 30331 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:32.826777 30276 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.828080 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:32.828766 29577 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:32.829252 29652 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:32.829840 29281 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:32.831334 30331 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.832664 29456 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:32.854223 29179 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
W20260812 19:06:32.855674 29528 consensus_peers.cc:487] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Couldn't send request to peer 182784ca1eb74135b8f16d8894fa4b01. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:32.879400 29176 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:32.912762 29649 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:32.913166 29649 ts_tablet_manager.cc:1397] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.161s user 0.015s sys 0.008s
I20260812 19:06:32.920322 29652 tablet_bootstrap.cc:654] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.912871 29178 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:32.925168 30331 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:32.927028 29650 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:32.927500 29650 ts_tablet_manager.cc:1397] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.153s user 0.005s sys 0.030s
I20260812 19:06:32.929061 29649 raft_consensus.cc:348] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.929852 29649 raft_consensus.cc:374] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.930158 29649 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.930671 29649 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.931912 30218 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:32.935184 30218 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.936188 29649 ts_tablet_manager.cc:1428] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260812 19:06:32.942516 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:32.943205 29459 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.943532 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:32.944170 29342 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:32.944236 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:32.943115 30218 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.942945 29650 raft_consensus.cc:348] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.946571 29650 raft_consensus.cc:374] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.947405 30159 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:32.947777 30159 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:32.947053 29650 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.948103 30159 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.948691 29650 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.950462 29652 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:32.950904 29652 ts_tablet_manager.cc:1397] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.122s user 0.007s sys 0.016s
I20260812 19:06:32.951460 29650 ts_tablet_manager.cc:1428] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.024s user 0.007s sys 0.001s
I20260812 19:06:32.956844 29650 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:32.966871 29649 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:32.973260 29650 tablet_bootstrap.cc:654] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:32.986384 30159 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.990119 29652 raft_consensus.cc:348] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:32.992707 29652 raft_consensus.cc:374] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:32.994640 29652 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:32.994928 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:32.996457 29459 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:32.994051 30159 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:32.995440 29652 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.002588 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:33.003535 29332 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.012988 29880 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:33.016547 29652 ts_tablet_manager.cc:1428] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.065s user 0.006s sys 0.000s
I20260812 19:06:33.023532 29652 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:33.020896 29459 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:33.037182 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:33.038479 29649 tablet_bootstrap.cc:654] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:33.038125 30276 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:33.039963 30276 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:33.046882 29650 tablet_bootstrap.cc:492] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:33.042125 30276 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.047380 29650 ts_tablet_manager.cc:1397] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.091s user 0.013s sys 0.004s
I20260812 19:06:33.051770 29332 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:33.086333 30331 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.087595 29650 raft_consensus.cc:348] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.087312 30331 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.088408 29650 raft_consensus.cc:374] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:33.088706 29650 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:33.090893 30331 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:33.092262 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:33.092692 30218 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:33.095671 29652 tablet_bootstrap.cc:654] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:33.089355 29650 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.101615 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:33.108258 29282 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:33.111872 30331 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:33.112941 29650 ts_tablet_manager.cc:1428] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.065s user 0.011s sys 0.000s
I20260812 19:06:33.117301 29650 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:33.117215 30278 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.131304 30278 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.133479 30278 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:33.135254 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:33.135922 29575 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:33.137007 29399 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:33.137614 29880 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:33.141294 30278 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:33.142350 30278 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:33.142486 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:33.142741 30278 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:33.146908 29403 leader_election.cc:341] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:33.148584 29530 consensus_peers.cc:487] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:33.147894 30177 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.149832 30177 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.151791 30177 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:33.157516 29176 catalog_manager.cc:5495] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:33.148216 30328 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.163235 29649 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:33.163864 29649 ts_tablet_manager.cc:1397] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.207s user 0.031s sys 0.031s
I20260812 19:06:33.165645 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:33.162035 30328 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.169579 30328 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:33.172159 29530 leader_election.cc:341] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:33.173321 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:33.173874 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:33.176231 30278 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.177366 30276 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.178081 30278 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:33.177809 30276 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.179145 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:33.179687 30276 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:33.180310 29649 raft_consensus.cc:348] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.180914 29649 raft_consensus.cc:374] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:33.181140 29649 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:33.180684 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:33.181697 29649 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.181751 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:33.185501 29649 ts_tablet_manager.cc:1428] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.021s user 0.005s sys 0.001s
I20260812 19:06:33.184677 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:33.186201 29530 leader_election.cc:341] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:33.187106 30328 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:33.183907 29572 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.188205 29649 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:33.188637 29650 tablet_bootstrap.cc:654] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:33.188789 29403 leader_election.cc:341] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:33.196648 29652 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:33.197084 29652 ts_tablet_manager.cc:1397] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.174s user 0.010s sys 0.020s
W20260812 19:06:33.197566 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 79844.8us
I20260812 19:06:33.199819 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:33.200482 29451 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:33.201061 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:33.204355 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:33.207206 30218 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:33.207588 30218 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:33.207921 30218 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.212496 30159 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:33.214993 29652 raft_consensus.cc:348] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.215693 29652 raft_consensus.cc:374] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:33.215929 29652 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:33.216473 29652 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.209743 29572 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
W20260812 19:06:33.222873 29528 leader_election.cc:341] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:33.223325 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:33.222688 29399 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:33.233880 29652 ts_tablet_manager.cc:1428] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.037s user 0.005s sys 0.001s
I20260812 19:06:33.232838 30328 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:33.229902 30426 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:33.265852 29652 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:33.266921 30328 raft_consensus.cc:686] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:33.269665 30328 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.267470 30177 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260812 19:06:33.282331 30218 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.292450 29456 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.309206 29650 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:33.309799 29650 ts_tablet_manager.cc:1397] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.196s user 0.006s sys 0.018s
I20260812 19:06:33.311030 29880 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:33.348667 30218 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:33.402015 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 63643.1us
W20260812 19:06:33.406279 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:33.378132 30426 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
W20260812 19:06:33.406924 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 43906.9us
I20260812 19:06:33.426563 29326 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.427372 30335 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.410816 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:33.452097 29530 leader_election.cc:341] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:33.454608 29650 raft_consensus.cc:348] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.455437 29650 raft_consensus.cc:374] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:33.455713 29650 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:33.457144 29650 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.429888 30335 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.499477 29649 tablet_bootstrap.cc:654] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:33.462960 30426 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260812 19:06:33.512612 29650 ts_tablet_manager.cc:1428] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.202s user 0.008s sys 0.000s
I20260812 19:06:33.522923 29652 tablet_bootstrap.cc:654] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:33.525473 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:33.516103 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:33.519531 30335 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:33.604449 29650 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:33.589442 29451 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:33.713239 30433 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.713860 30433 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.805624 29176 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:33.809289 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:33.855404 29650 tablet_bootstrap.cc:654] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:33.886932 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:33.869851 30426 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:33.907096 30426 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:33.968149 29282 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:33.969208 30454 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260812 19:06:33.987002 29403 leader_election.cc:341] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:33.989708 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 434889us
I20260812 19:06:33.990149 30470 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:34.000221 30433 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:34.002940 29403 consensus_peers.cc:487] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:34.007208 29880 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:34.036605 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:34.092681 29650 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:34.093184 29650 ts_tablet_manager.cc:1397] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.506s user 0.017s sys 0.019s
I20260812 19:06:34.067951 29342 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:34.101382 29650 raft_consensus.cc:348] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.093511 30426 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:34.104535 29650 raft_consensus.cc:374] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:34.105803 29650 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
W20260812 19:06:34.089674 30449 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.052s user 0.001s sys 0.001s
I20260812 19:06:34.085606 29579 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
W20260812 19:06:34.119549 29451 consensus_meta.cc:287] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.507s user 0.009s sys 0.004s
I20260812 19:06:34.120177 29451 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:34.122040 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:34.136003 30460 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:34.136804 30460 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:34.127348 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:34.142333 29528 leader_election.cc:341] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:34.148680 29650 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.150736 29650 ts_tablet_manager.cc:1428] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.057s user 0.007s sys 0.003s
I20260812 19:06:34.154626 29650 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:34.159395 30460 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:34.189360 30464 log.cc:924] Time spent T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.258s user 0.005s sys 0.000s
W20260812 19:06:34.197232 30466 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.250s user 0.002s sys 0.000s
I20260812 19:06:34.256485 30433 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:34.257333 30433 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.272290 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:34.278750 29403 leader_election.cc:341] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:34.285678 29650 tablet_bootstrap.cc:654] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:34.286842 30159 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:34.261720 30433 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:34.376151 29649 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:34.376634 29649 ts_tablet_manager.cc:1397] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 1.189s user 0.018s sys 0.050s
I20260812 19:06:34.392655 29649 raft_consensus.cc:348] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.393425 29649 raft_consensus.cc:374] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:34.393692 29649 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:34.399711 29649 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.429909 30276 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:34.430585 30276 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:34.446619 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 408600us
I20260812 19:06:34.457298 29649 ts_tablet_manager.cc:1428] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.080s user 0.008s sys 0.002s
W20260812 19:06:34.462303 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 426969us
I20260812 19:06:34.490345 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:34.472945 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 319) took 1017 ms. Trace:
I20260812 19:06:34.504838 30440 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260812 19:06:34.521697 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:34.492156 30276 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:34.510802 29649 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:34.536245 29530 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:34.552707 30472 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:34.513026 29331 rpcz_store.cc:276] 0812 19:06:33.453404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:33.456387 (+ 2983us) service_pool.cc:227] Handling call
0812 19:06:33.467988 (+ 11601us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:06:33.808034 (+340046us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:06:33.808070 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:06:33.864374 (+ 56304us) log.cc:842] Serialized 26 byte log entry
0812 19:06:34.029862 (+165488us) raft_consensus.cc:1631] Marking committed up to 1
0812 19:06:34.374750 (+344888us) log.cc:842] Serialized 14 byte log entry
0812 19:06:34.388310 (+ 13560us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b240017a908. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66d4acf4 00007f9c66d48dfb 00007f9c66dd7559 00007f9c66dd71e5 00007f9c66dd682d 00007f9c69a7946e 00007f9c69a90c0c 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551
0812 19:06:34.406669 (+ 18359us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:06:34.426137 (+ 19468us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:06:34.429744 (+ 3607us) raft_consensus.cc:1671] finished
0812 19:06:34.453892 (+ 24148us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:06:34.470825 (+ 16933us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34167,"prepare.queue_time_us":99263,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":76325,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":21806,"spinlock_wait_cycles":6448384,"thread_start_us":762795,"threads_started":3,"wal-append.queue_time_us":215294}
I20260812 19:06:34.659266 30443 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:34.659847 30443 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.664350 30443 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:06:34.688509 29339 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s user 0.000s sys 0.051s
W20260812 19:06:34.783212 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 226606us
W20260812 19:06:34.785811 29326 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.825s user 0.029s sys 0.013s
I20260812 19:06:34.689327 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:34.781756 29334 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s user 0.010s sys 0.023s
W20260812 19:06:34.706010 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.623s user 0.002s sys 0.045s
W20260812 19:06:34.725975 30460 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s user 0.015s sys 0.022s
I20260812 19:06:34.682181 30458 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:34.805662 30458 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.809409 30458 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:34.736580 30482 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.024s sys 0.024s
W20260812 19:06:34.803828 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.730s user 0.002s sys 0.045s
W20260812 19:06:34.836555 30482 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.608s user 0.029s sys 0.030s
W20260812 19:06:34.814265 30460 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s user 0.016s sys 0.022s
W20260812 19:06:34.810484 29334 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s user 0.010s sys 0.024s
W20260812 19:06:34.814733 29339 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s user 0.001s sys 0.051s
W20260812 19:06:34.803390 29652 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.847s user 0.113s sys 0.142s
I20260812 19:06:34.816237 29649 tablet_bootstrap.cc:654] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:34.838850 29652 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.884s user 0.113s sys 0.143s
W20260812 19:06:34.804235 29326 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.859s user 0.031s sys 0.013s
I20260812 19:06:34.840155 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:34.844347 29652 tablet_bootstrap.cc:492] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:34.845692 29652 ts_tablet_manager.cc:1397] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 1.599s user 0.140s sys 0.164s
I20260812 19:06:34.849114 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 315) took 1438 ms. Trace:
I20260812 19:06:34.856205 29650 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:34.857177 29650 ts_tablet_manager.cc:1397] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.703s user 0.021s sys 0.042s
I20260812 19:06:34.859840 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:34.862138 30433 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:34.862583 30433 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:34.865973 30433 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:34.867256 29650 raft_consensus.cc:348] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.868131 29650 raft_consensus.cc:374] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:34.868454 29650 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:34.869302 29650 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.871296 29650 ts_tablet_manager.cc:1428] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260812 19:06:34.872912 29650 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:34.849677 29326 rpcz_store.cc:276] 0812 19:06:33.410657 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:33.418914 (+ 8257us) service_pool.cc:227] Handling call
0812 19:06:33.426396 (+ 7482us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:06:33.690910 (+264514us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:06:33.692220 (+ 1310us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:06:33.784823 (+ 92603us) log.cc:842] Serialized 26 byte log entry
0812 19:06:33.881376 (+ 96553us) raft_consensus.cc:1631] Marking committed up to 1
0812 19:06:34.848087 (+966711us) log.cc:842] Serialized 14 byte log entry
0812 19:06:34.848622 (+ 535us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:06:34.848741 (+ 119us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:06:34.848819 (+ 78us) raft_consensus.cc:1671] finished
0812 19:06:34.848833 (+ 14us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:06:34.849021 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":67180,"prepare.queue_time_us":220665,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":12661,"raft.queue_time_us":78638,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":4044,"thread_start_us":977476,"threads_started":2,"wal-append.queue_time_us":166949}
I20260812 19:06:34.895646 30456 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:34.902083 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:34.903066 30456 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.872939 29652 raft_consensus.cc:348] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.908823 29652 raft_consensus.cc:374] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:34.909140 29652 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:34.909751 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:34.926682 29649 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:34.927511 29649 ts_tablet_manager.cc:1397] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.451s user 0.009s sys 0.012s
I20260812 19:06:34.908031 30456 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:34.950445 29652 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.952346 29652 ts_tablet_manager.cc:1428] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.106s user 0.005s sys 0.001s
I20260812 19:06:34.947887 29649 raft_consensus.cc:348] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.954420 29649 raft_consensus.cc:374] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:34.954734 29649 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:34.955385 29649 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:34.957231 29649 ts_tablet_manager.cc:1428] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.029s user 0.004s sys 0.000s
I20260812 19:06:34.953837 29652 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:34.968549 29650 tablet_bootstrap.cc:654] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:34.944981 30433 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.207810 30455 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.208576 30455 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.211442 30455 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.224331 29649 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:35.229295 30510 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.230094 30510 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.232151 30177 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.232267 30510 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.232848 30177 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.235023 30177 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.235742 29649 tablet_bootstrap.cc:654] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.247532 29649 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:35.247977 29649 ts_tablet_manager.cc:1397] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.004s
I20260812 19:06:35.250598 29649 raft_consensus.cc:348] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.251376 29649 raft_consensus.cc:374] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.251641 29649 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.252470 29649 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.254169 29649 ts_tablet_manager.cc:1428] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 19:06:35.255483 29649 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Bootstrap starting.
I20260812 19:06:35.261700 29649 tablet_bootstrap.cc:654] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.265774 30177 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.266512 30177 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.268636 30177 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.272646 30433 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.274869 29649 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: No bootstrap required, opened a new log
I20260812 19:06:35.275229 29650 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:35.275391 29649 ts_tablet_manager.cc:1397] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260812 19:06:35.275640 29650 ts_tablet_manager.cc:1397] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.403s user 0.013s sys 0.004s
I20260812 19:06:35.278193 29650 raft_consensus.cc:348] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.278342 29649 raft_consensus.cc:348] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.278913 29650 raft_consensus.cc:374] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.279201 29650 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.279879 29650 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.280733 30331 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.281423 29650 ts_tablet_manager.cc:1428] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 19:06:35.281235 30331 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.279206 29649 raft_consensus.cc:374] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.282138 29649 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.282440 29650 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:35.282979 29649 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.285050 29649 ts_tablet_manager.cc:1428] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260812 19:06:35.289177 30331 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.291203 29650 tablet_bootstrap.cc:654] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.294941 30177 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.295615 30177 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.298285 30177 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.317345 30330 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.318572 30330 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.320616 30330 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.329166 29652 tablet_bootstrap.cc:654] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.345525 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:06:35.357378 29528 leader_election.cc:341] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.375602 29650 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:35.375075 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:35.376154 29650 ts_tablet_manager.cc:1397] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.094s user 0.024s sys 0.031s
I20260812 19:06:35.376893 29575 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.382663 29650 raft_consensus.cc:348] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.383486 29650 raft_consensus.cc:374] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.383756 29650 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.384503 29650 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.387287 29650 ts_tablet_manager.cc:1428] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260812 19:06:35.388767 29650 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Bootstrap starting.
I20260812 19:06:35.398839 29652 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:35.399264 29652 ts_tablet_manager.cc:1397] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.446s user 0.022s sys 0.022s
I20260812 19:06:35.404212 29652 raft_consensus.cc:348] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.405148 29652 raft_consensus.cc:374] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.405501 29652 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
W20260812 19:06:35.406375 29399 leader_election.cc:341] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.406224 29652 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.406843 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.407502 29650 tablet_bootstrap.cc:654] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.409353 29652 ts_tablet_manager.cc:1428] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260812 19:06:35.410244 30457 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:35.411970 29652 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:35.424924 30455 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.427106 30455 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.432148 30455 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.435837 29575 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:35.439972 29652 tablet_bootstrap.cc:654] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.444924 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.446053 30451 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:35.446566 30451 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:35.447531 30451 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.448678 29880 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.450161 29880 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.452286 29880 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.458432 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.467574 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 97) took 1998 ms. Trace:
I20260812 19:06:35.468583 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.469416 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.470573 30452 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.472285 30159 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:35.471951 30452 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.474817 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:35.476881 29530 leader_election.cc:341] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.477428 29530 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.483024 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.485183 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.487433 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.488847 29572 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:06:35.489807 29257 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260812 19:06:35.491039 30462 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.492882 30448 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:35.492089 30462 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.473472 30159 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.481474 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.500442 30159 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.507465 29567 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:35.514333 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.515579 29178 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55498 (request call id 106) took 2047 ms. Trace:
I20260812 19:06:35.515995 29650 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: No bootstrap required, opened a new log
I20260812 19:06:35.515890 30448 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.516588 29650 ts_tablet_manager.cc:1397] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Time spent bootstrapping tablet: real 0.128s user 0.013s sys 0.009s
I20260812 19:06:35.516641 30448 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.515959 29178 rpcz_store.cc:276] 0812 19:06:33.478280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:33.604219 (+125939us) service_pool.cc:227] Handling call
0812 19:06:35.514614 (+1910395us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":167021,"reported_tablets":5}
I20260812 19:06:35.520749 30448 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.503849 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.522666 29650 raft_consensus.cc:348] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.523520 29650 raft_consensus.cc:374] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.523808 29650 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.528337 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.506642 30462 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.534013 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.524559 29650 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.538040 30455 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:35.538421 30462 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.538970 30462 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.538666 30455 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.544827 30455 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.540539 30452 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.555991 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.557090 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.560321 30462 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.541570 30448 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.570322 30448 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.570845 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.572561 30448 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.574265 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.574721 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.468101 29176 rpcz_store.cc:276] 0812 19:06:33.486692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:33.489797 (+ 3105us) service_pool.cc:227] Handling call
0812 19:06:35.466548 (+1976751us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:06:33.915280 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:06:33.916471 (+ 1191us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:06:33.916515 (+ 44us) write_op.cc:435] Acquired schema lock
0812 19:06:33.916529 (+ 14us) tablet.cc:660] Decoding operations
0812 19:06:33.917189 (+ 660us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:06:33.917315 (+ 126us) write_op.cc:641] Partition lock acquired for write op
0812 19:06:33.917368 (+ 53us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:06:33.923216 (+ 5848us) tablet.cc:699] Row locks acquired
0812 19:06:33.923233 (+ 17us) write_op.cc:260] PREPARE: finished
0812 19:06:33.923527 (+ 294us) write_op.cc:270] Start()
0812 19:06:33.923735 (+ 208us) write_op.cc:276] Timestamp: P: 1786561593923452 usec, L: 0
0812 19:06:33.923758 (+ 23us) op_driver.cc:331] REPLICATION: starting
0812 19:06:33.924757 (+ 999us) log.cc:842] Serialized 540 byte log entry
0812 19:06:34.214886 (+290129us) op_driver.cc:444] REPLICATION: finished
0812 19:06:34.816369 (+601483us) write_op.cc:301] APPLY: starting
0812 19:06:34.816566 (+ 197us) tablet.cc:1365] starting BulkCheckPresence
0812 19:06:34.816816 (+ 250us) tablet.cc:1368] finished BulkCheckPresence
0812 19:06:34.817531 (+ 715us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:06:34.818088 (+ 557us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:06:34.818438 (+ 350us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:06:34.818538 (+ 100us) write_op.cc:312] APPLY: finished
0812 19:06:34.819055 (+ 517us) log.cc:842] Serialized 22 byte log entry
0812 19:06:34.843633 (+ 24578us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:06:34.858792 (+ 15159us) write_op.cc:454] Released schema lock
0812 19:06:34.858870 (+ 78us) write_op.cc:341] FINISH: Updating metrics
0812 19:06:34.859263 (+ 393us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:06:34.859289 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":596332,"apply.run_cpu_time_us":3805,"apply.run_wall_time_us":43347,"num_ops":1,"prepare.queue_time_us":104915,"prepare.run_cpu_time_us":4442,"prepare.run_wall_time_us":9856,"replication_time_us":277964,"thread_start_us":666651,"threads_started":2}]]}
I20260812 19:06:35.601326 30462 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.601485 29650 ts_tablet_manager.cc:1428] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Time spent starting tablet: real 0.084s user 0.008s sys 0.001s
I20260812 19:06:35.640718 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.641405 29456 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:35.642187 30159 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.642699 30159 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.638403 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.645355 30433 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:35.648963 30433 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:35.648727 30462 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.638065 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.649343 30433 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.633157 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.630831 30177 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.650717 30177 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.651497 30462 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.657603 30177 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.660809 30159 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.662365 30448 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:35.664775 29652 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:35.665261 29652 ts_tablet_manager.cc:1397] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.255s user 0.019s sys 0.008s
I20260812 19:06:35.665334 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.666703 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.666882 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.665269 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.668318 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:35.672417 29337 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.672349 29652 raft_consensus.cc:348] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.676234 29652 raft_consensus.cc:374] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.676549 29652 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.677037 29282 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.677191 29652 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.678222 30489 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.678916 29282 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.680150 29282 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.681260 30452 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.680316 30489 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.684477 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.685434 30283 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:35.685715 30283 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:35.685988 30283 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.671559 30433 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.688639 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.689406 29452 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 0.
I20260812 19:06:35.690419 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.692289 29282 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.693008 30455 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.693826 30455 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.694998 29652 ts_tablet_manager.cc:1428] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.025s user 0.005s sys 0.001s
I20260812 19:06:35.695207 29282 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.696142 29652 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:35.696620 30439 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.699005 29341 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.701730 30455 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.707633 30283 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.716548 29528 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.717315 30448 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:35.717715 30448 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:35.718102 30448 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.721261 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.728595 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.720286 30439 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:06:35.724785 29178 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:35.733924 30439 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.750439 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:35.751019 29578 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.757970 29652 tablet_bootstrap.cc:654] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.744532 30433 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.761036 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:35.769447 29337 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:35.760318 30283 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.759629 30448 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.770406 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.772643 29452 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.759980 29578 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:35.778115 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.778316 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.774649 30489 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:35.781068 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.781801 30433 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:35.782490 30433 raft_consensus.cc:686] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:35.783334 30433 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.775410 30455 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:35.794292 30455 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.796267 30455 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.797397 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.798141 30439 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.792558 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.800732 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.804550 30457 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:35.805073 30457 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.807118 30457 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.807222 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.807883 29339 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:35.809322 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.811482 29452 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:35.814312 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.814913 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.815136 30177 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:35.815618 30177 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:35.816702 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.816532 30177 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.819335 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.822517 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.825039 30448 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.835009 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:35.835777 29445 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.851011 29652 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:35.851570 29652 ts_tablet_manager.cc:1397] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.156s user 0.013s sys 0.007s
I20260812 19:06:35.847949 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:35.855777 29652 raft_consensus.cc:348] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.856506 29652 raft_consensus.cc:374] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:35.856783 29652 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:35.858683 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:35.862576 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.863834 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.857372 29652 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.866000 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:06:35.869678 29403 leader_election.cc:341] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:35.871464 29530 leader_election.cc:341] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.875798 29530 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.876760 29403 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.873072 29445 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:06:35.874275 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.874977 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.875227 29652 ts_tablet_manager.cc:1428] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.023s user 0.003s sys 0.003s
I20260812 19:06:35.877437 30159 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:35.871974 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.878353 29530 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.885479 29530 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.886950 29530 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.887949 29339 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:35.889824 29530 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:35.890980 30474 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:06:35.890937 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:06:35.892421 29530 leader_election.cc:341] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:35.893072 29530 leader_election.cc:341] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.893898 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.883992 30457 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.894956 29530 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:35.897389 30460 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.887524 30159 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.891809 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.898680 29528 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:35.891585 30510 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.896468 30494 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.899655 30510 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:35.900084 30510 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:35.900489 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:35.900875 30510 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.891686 30177 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.894377 30283 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.903512 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:35.907030 29403 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.908566 30457 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.891381 30474 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:35.911382 30474 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:35.915186 29652 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:35.898360 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:35.925052 30457 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:35.935638 29282 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.938163 30439 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.940201 29652 tablet_bootstrap.cc:654] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:35.940984 29282 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.943737 30439 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.944967 29282 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.946029 30474 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.947877 29578 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:06:35.949782 30474 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:35.950604 30439 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.951573 30433 consensus_queue.cc:1046] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:35.953784 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:35.957980 29282 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.960428 30439 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.961529 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.964516 30439 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.965790 29282 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.969905 30439 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.971290 29282 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:35.953337 30457 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
W20260812 19:06:35.971961 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 74990us
I20260812 19:06:35.972460 29282 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:06:35.973100 29530 leader_election.cc:341] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.972532 30489 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:35.974089 30330 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:35.976433 30444 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:06:35.979527 29530 leader_election.cc:341] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:35.979132 30444 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:06:35.983847 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080003ef00 after lost signal to thread 27450
W20260812 19:06:35.984380 29403 leader_election.cc:341] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
W20260812 19:06:35.984884 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080003d5e0 after lost signal to thread 29236
I20260812 19:06:35.987185 30444 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:35.972373 30457 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:35.987425 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:35.988179 29578 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:35.988294 30457 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:35.989591 30457 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:35.990499 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:06:35.995270 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08002a1ea0 after lost signal to thread 29379
I20260812 19:06:35.995667 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:06:35.996243 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08002a1000 after lost signal to thread 29496
W20260812 19:06:35.997294 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080026fb40 after lost signal to thread 29622
I20260812 19:06:36.002329 30444 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:36.003822 30444 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:36.004172 30444 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
W20260812 19:06:36.010022 29403 leader_election.cc:341] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:36.014833 30433 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:36.021777 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:36.023226 30278 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:36.023803 30278 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.025874 29530 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:36.025940 30278 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.027850 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:36.032572 30550 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:06:36.036073 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:36.037783 29567 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:36.038882 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:36.039557 29338 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 0.
I20260812 19:06:36.040172 30278 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 19:06:36.040627 30278 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:36.040997 30278 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.045804 29652 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:36.046312 29652 ts_tablet_manager.cc:1397] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.158s user 0.026s sys 0.022s
I20260812 19:06:36.056679 30444 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.057324 29652 raft_consensus.cc:348] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.058063 29652 raft_consensus.cc:374] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:36.058348 29652 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:36.059103 29652 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.061607 29652 ts_tablet_manager.cc:1428] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260812 19:06:36.062566 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:36.062839 29652 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:36.063231 29567 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.061556 30444 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.066025 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:06:36.071538 29403 leader_election.cc:341] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Illegal state: must be running to vote when last-logged opid is not known
I20260812 19:06:36.070042 30278 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.075178 30278 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.076092 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:36.076270 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:36.076735 29339 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.076691 29578 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.083140 29339 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:36.083518 29578 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:36.084491 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:36.085072 30278 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:36.085726 30278 raft_consensus.cc:686] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:36.086705 30278 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.109344 29652 tablet_bootstrap.cc:654] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:36.114372 29176 catalog_manager.cc:5495] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:36.119001 29567 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:06:36.125123 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:36.136796 29652 tablet_bootstrap.cc:492] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:36.128062 30278 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 19:06:36.137333 29652 ts_tablet_manager.cc:1397] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.075s user 0.008s sys 0.002s
I20260812 19:06:36.138618 29179 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:36.137708 30278 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:36.140236 29652 raft_consensus.cc:348] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.140511 29179 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 0 to 1, leader changed from <none> to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:06:36.141654 29652 raft_consensus.cc:374] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:36.141937 29652 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:36.142587 29652 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.142731 30278 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.144785 29652 ts_tablet_manager.cc:1428] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260812 19:06:36.146759 29652 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:36.152185 29652 tablet_bootstrap.cc:654] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
W20260812 19:06:36.166477 29403 consensus_peers.cc:487] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Couldn't send request to peer 6dd313c6592d4f33bab314a73f28b007. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:36.179345 29652 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:36.179841 29652 ts_tablet_manager.cc:1397] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.005s
I20260812 19:06:36.183051 29652 raft_consensus.cc:348] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.183735 29652 raft_consensus.cc:374] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:36.183965 29652 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:36.184523 29652 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.186640 29652 ts_tablet_manager.cc:1428] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:06:36.199453 29652 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Bootstrap starting.
I20260812 19:06:36.217702 29178 catalog_manager.cc:5495] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:36.219576 29178 catalog_manager.cc:5495] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:36.220995 29178 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:36.217710 29652 tablet_bootstrap.cc:654] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Neither blocks nor log segments found. Creating new log.
I20260812 19:06:36.228928 30330 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:36.229434 30330 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.230860 30331 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:36.231683 30330 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.232882 30331 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.233664 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:36.234583 29457 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:06:36.238375 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:36.243620 29282 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:36.244920 30331 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.245419 30330 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.251909 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:36.260072 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:36.260848 29578 raft_consensus.cc:2379] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:36.268911 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:36.275072 30462 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.280679 30330 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:36.282728 29652 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: No bootstrap required, opened a new log
I20260812 19:06:36.283233 29652 ts_tablet_manager.cc:1397] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Time spent bootstrapping tablet: real 0.084s user 0.021s sys 0.013s
I20260812 19:06:36.284202 30218 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:36.286139 29652 raft_consensus.cc:348] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.286823 29652 raft_consensus.cc:374] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 19:06:36.287250 29652 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Initialized, Role: FOLLOWER
I20260812 19:06:36.287833 29652 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.290108 29652 ts_tablet_manager.cc:1428] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 19:06:36.308955 30462 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:36.318276 30462 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:36.318691 30462 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.320124 30462 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.321174 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:36.322003 29578 raft_consensus.cc:2379] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 1: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:06:36.322750 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:36.326038 30218 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:36.329177 29282 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:36.330026 30455 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.340224 30455 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 19:06:36.395246 30218 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:36.398288 29341 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 0 FOLLOWER]: Advancing to term 1
I20260812 19:06:36.413079 30278 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:36.414304 30455 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:36.414948 30455 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:36.424821 30218 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:36.425972 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:36.428552 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:36.426841 30455 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:36.432974 29282 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:36.446759 30439 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:36.542745 30457 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 19:06:36.567814 30278 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:36.681440 30456 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:36.747625 30457 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 19:06:37.185083 30433 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:06:37.185587 30433 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:37.187644 30433 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:37.190603 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:37.191257 29578 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:06:37.194341 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:37.195066 30444 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:06:37.195463 30444 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:06:37.195767 30444 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:37.199719 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:37.202489 30444 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:37.204782 30444 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:37.205826 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:37.208647 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:37.209255 29576 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:06:37.215600 29576 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:06:37.216507 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:37.217280 30444 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:06:37.218788 30444 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:37.219580 30444 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:37.232317 29176 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:37.492542 30330 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:06:37.493144 30330 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:37.501029 27445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 19:06:37.506222 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:06:37.509613 30330 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:37.520726 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:37.521646 29576 raft_consensus.cc:2354] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3.
I20260812 19:06:37.524308 29281 leader_election.cc:398] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3.
I20260812 19:06:37.524616 29281 leader_election.cc:401] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:06:37.526839 30596 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 3
W20260812 19:06:37.535239 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:37.556507 30596 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3.
I20260812 19:06:37.558694 27445 server_base.cc:1034] running on GCE node
W20260812 19:06:37.559047 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 19:06:37.558295 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 19:06:37.563508 27445 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 19:06:37.564492 27445 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 19:06:37.564841 27445 hybrid_clock.cc:648] HybridClock initialized: now 1786561597564820 us; error 0 us; skew 500 ppm
I20260812 19:06:37.565683 27445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 19:06:37.591557 27445 webserver.cc:460] Webserver started at http://127.26.205.68:34369/ using document root <none> and password file <none>
I20260812 19:06:37.592127 27445 fs_manager.cc:359] Metadata directory not provided
I20260812 19:06:37.592336 27445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 19:06:37.592717 27445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 19:06:37.594129 27445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-3-root/instance:
uuid: "a34c47dd29c14748a018d716ac6c39df"
format_stamp: "Formatted at 2026-08-12 19:06:37 on dist-test-slave-vpvm"
I20260812 19:06:37.600708 27445 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260812 19:06:37.609043 29576 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 19:06:37.611770 30444 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:37.691798 29339 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 19:06:37.695111 30612 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:37.798882 30609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:37.803259 27445 fs_manager.cc:725] Time spent opening block manager: real 0.200s user 0.010s sys 0.052s
I20260812 19:06:37.803846 27445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-3-root
uuid: "a34c47dd29c14748a018d716ac6c39df"
format_stamp: "Formatted at 2026-08-12 19:06:37 on dist-test-slave-vpvm"
I20260812 19:06:37.804344 27445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 19:06:37.888298 27445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 19:06:37.890153 27445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 19:06:37.902412 27445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 19:06:37.922909 27445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 19:06:37.923317 27445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:37.923687 27445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 19:06:37.923892 27445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 19:06:38.384027 30721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.205.68:41925 every 8 connection(s)
I20260812 19:06:38.384615 27445 rpc_server.cc:307] RPC server started. Bound to: 127.26.205.68:41925
I20260812 19:06:38.440474 30724 heartbeater.cc:347] Connected to a master server at 127.26.205.126:35929
I20260812 19:06:38.441171 30724 heartbeater.cc:464] Registering TS with master...
I20260812 19:06:38.442229 30724 heartbeater.cc:511] Master 127.26.205.126:35929 requested a full tablet report, sending...
I20260812 19:06:38.444491 29176 ts_manager.cc:194] Registered new tserver with Master: a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:06:38.447086 29176 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53840
I20260812 19:06:38.607098 27445 auto_leader_rebalancer-test.cc:413] add a tserver: a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:39.054617 29341 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:39.073870 29576 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.075769 30596 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:39.089123 29452 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.092228 30439 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260812 19:06:39.120265 30745 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260812 19:06:39.126246 30439 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.131774 29452 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.151865 29576 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.154038 29176 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:39.158463 29281 consensus_peers.cc:487] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a5c317250947485cab086b4bfab851bd. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:39.229591 29457 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:39.265635 29576 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.269336 29331 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.271569 30737 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:39.273532 30736 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:39.303889 30736 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.305840 29576 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.310881 29331 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:39.322119 29399 consensus_peers.cc:487] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2743b6275ccc4fbdbe5f71c667adee8b. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:39.337337 29331 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:06:39.364280 29281 consensus_peers.cc:487] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 637fa5c7fa824870a0da61e2bc8400af. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:39.366806 29457 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.366681 29178 catalog_manager.cc:5495] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.368065 29576 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.375694 30330 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:39.377831 30330 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:39.390616 30330 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.400684 29457 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.426223 29576 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:39.427371 29178 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:39.451715 30724 heartbeater.cc:503] Master 127.26.205.126:35929 was elected leader, sending a full tablet report...
I20260812 19:06:39.465986 29576 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:39.499656 29457 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.501358 30218 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:39.501338 29331 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:39.503701 30177 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:39.618552 30782 log.cc:924] Time spent T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.062s user 0.000s sys 0.006s
I20260812 19:06:39.620285 30786 ts_tablet_manager.cc:927] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
W20260812 19:06:39.625344 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 58323.4us
I20260812 19:06:39.865749 30177 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:39.998986 29527 consensus_peers.cc:487] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 14e182b70d344161a275142a344cd1ba. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:39.933317 30786 tablet_copy_client.cc:316] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:06:40.016588 29459 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:40.136109 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 217991us
I20260812 19:06:40.160953 30795 ts_tablet_manager.cc:927] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:06:40.239054 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 72204.1us
I20260812 19:06:40.343606 30814 ts_tablet_manager.cc:927] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:40.353345 30795 tablet_copy_client.cc:316] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:40.368314 29331 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:40.390646 29176 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:40.374068 30821 ts_tablet_manager.cc:927] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:40.401841 30814 tablet_copy_client.cc:316] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:40.425864 29358 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet a5c317250947485cab086b4bfab851bd from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:06:40.452756 29358 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet a5c317250947485cab086b4bfab851bd from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-a5c317250947485cab086b4bfab851bd
I20260812 19:06:40.481003 30821 tablet_copy_client.cc:316] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:06:40.490226 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 637fa5c7fa824870a0da61e2bc8400af from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:06:40.491447 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 637fa5c7fa824870a0da61e2bc8400af from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-637fa5c7fa824870a0da61e2bc8400af
I20260812 19:06:40.527297 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 2743b6275ccc4fbdbe5f71c667adee8b from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:40.550798 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 2743b6275ccc4fbdbe5f71c667adee8b from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-2743b6275ccc4fbdbe5f71c667adee8b
I20260812 19:06:40.566128 29358 tablet_copy_source_session.cc:205] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:40.580282 29355 tablet_copy_source_session.cc:205] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:40.592607 30786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5c317250947485cab086b4bfab851bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:40.612636 30821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 637fa5c7fa824870a0da61e2bc8400af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:40.659883 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 14e182b70d344161a275142a344cd1ba from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:40.661738 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 14e182b70d344161a275142a344cd1ba from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-14e182b70d344161a275142a344cd1ba
I20260812 19:06:40.673628 29601 tablet_copy_source_session.cc:205] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:40.678469 29474 tablet_copy_source_session.cc:205] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:40.687371 30814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14e182b70d344161a275142a344cd1ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:40.726112 30786 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:34276 (request call id 2) took 1233 ms. Trace:
I20260812 19:06:40.730648 30786 rpcz_store.cc:276] 0812 19:06:39.492440 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:39.492635 (+ 195us) service_pool.cc:227] Handling call
0812 19:06:39.624255 (+131620us) ts_tablet_manager.cc:928] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
0812 19:06:40.588770 (+964515us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:06:40.678600 (+ 89830us) tablet_metadata.cc:641] Metadata flushed
0812 19:06:40.678865 (+ 265us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:06:40.724255 (+ 45390us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:06:40.724672 (+ 417us) tablet_replica.cc:205] Creating consensus instance
0812 19:06:40.725941 (+ 1269us) ts_tablet_manager.cc:983] Replica registered.
0812 19:06:40.726052 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":85676,"tablet-copy.queue_time_us":126121,"thread_start_us":298228,"threads_started":2}
I20260812 19:06:40.734556 30786 tablet_copy_client.cc:791] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:40.735337 30814 tablet_copy_client.cc:791] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:40.736053 30814 tablet_copy_client.cc:655] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:40.735312 30786 tablet_copy_client.cc:655] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:40.748224 30786 tablet_copy_client.cc:531] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:40.758833 30821 tablet_copy_client.cc:791] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:40.759496 30821 tablet_copy_client.cc:655] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:40.760365 30814 tablet_copy_client.cc:531] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:40.765386 30795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2743b6275ccc4fbdbe5f71c667adee8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:40.766366 30786 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:40.782088 30821 tablet_copy_client.cc:531] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:40.819177 30821 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:40.836272 30795 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:34288 (request call id 1) took 1102 ms. Trace:
I20260812 19:06:40.842571 30814 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:40.847692 30795 rpcz_store.cc:276] 0812 19:06:39.733630 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:06:39.734942 (+ 1312us) service_pool.cc:227] Handling call
0812 19:06:40.187235 (+452293us) ts_tablet_manager.cc:928] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
0812 19:06:40.763789 (+576554us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:06:40.812259 (+ 48470us) tablet_metadata.cc:641] Metadata flushed
0812 19:06:40.812604 (+ 345us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:06:40.835041 (+ 22437us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:06:40.835539 (+ 498us) tablet_replica.cc:205] Creating consensus instance
0812 19:06:40.836100 (+ 561us) ts_tablet_manager.cc:983] Replica registered.
0812 19:06:40.836179 (+ 79us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":2302,"tablet-copy.queue_time_us":389182,"thread_start_us":232944,"threads_started":2}
I20260812 19:06:40.863185 30795 tablet_copy_client.cc:791] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:40.865639 30795 tablet_copy_client.cc:655] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:40.895706 30795 tablet_copy_client.cc:531] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:40.970329 30795 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:41.070534 30786 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:41.075286 30786 tablet_bootstrap.cc:492] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:41.076184 30786 ts_tablet_manager.cc:1397] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.318s user 0.029s sys 0.032s
I20260812 19:06:41.080824 30786 raft_consensus.cc:348] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.082495 30786 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:41.083866 30786 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.118484 30821 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:41.123112 30786 ts_tablet_manager.cc:1428] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.046s user 0.003s sys 0.010s
I20260812 19:06:41.126246 30821 tablet_bootstrap.cc:492] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:41.129420 30821 ts_tablet_manager.cc:1397] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.311s user 0.011s sys 0.038s
I20260812 19:06:41.132442 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-a5c317250947485cab086b4bfab851bd received from {username='slave'} at 127.0.0.1:41568
I20260812 19:06:41.133100 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-a5c317250947485cab086b4bfab851bd on tablet a5c317250947485cab086b4bfab851bd with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:41.132221 30814 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:41.165845 30814 tablet_bootstrap.cc:492] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:41.168009 30814 ts_tablet_manager.cc:1397] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.326s user 0.021s sys 0.025s
I20260812 19:06:41.175753 30795 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:41.176782 30795 tablet_bootstrap.cc:492] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:41.177544 30795 ts_tablet_manager.cc:1397] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.210s user 0.023s sys 0.040s
I20260812 19:06:41.181792 30821 raft_consensus.cc:348] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.182901 30821 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:41.183641 30821 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.180544 30814 raft_consensus.cc:348] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.187362 30814 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:41.188037 30814 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.193858 30814 ts_tablet_manager.cc:1428] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.020s user 0.005s sys 0.001s
I20260812 19:06:41.196458 30795 raft_consensus.cc:348] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.201831 30795 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:41.204903 30795 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:41.214715 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-14e182b70d344161a275142a344cd1ba received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:41.215152 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-14e182b70d344161a275142a344cd1ba on tablet 14e182b70d344161a275142a344cd1ba with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:41.216267 30795 ts_tablet_manager.cc:1428] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.038s user 0.006s sys 0.001s
I20260812 19:06:41.218166 30821 ts_tablet_manager.cc:1428] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.088s user 0.002s sys 0.007s
I20260812 19:06:41.219063 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-2743b6275ccc4fbdbe5f71c667adee8b received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:41.235953 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-2743b6275ccc4fbdbe5f71c667adee8b on tablet 2743b6275ccc4fbdbe5f71c667adee8b with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:41.252941 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-637fa5c7fa824870a0da61e2bc8400af received from {username='slave'} at 127.0.0.1:41568
I20260812 19:06:41.253372 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-637fa5c7fa824870a0da61e2bc8400af on tablet 637fa5c7fa824870a0da61e2bc8400af with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:41.445645 30686 raft_consensus.cc:1203] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:41.555912 30686 raft_consensus.cc:1203] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:41.594280 30686 raft_consensus.cc:1203] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:41.724619 30686 raft_consensus.cc:1203] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:41.926821 30587 raft_consensus.cc:1050] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:41.929526 30587 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:41.940392 29453 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:41.943280 30812 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:41.953632 29339 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:41.957170 30686 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:41.957764 30812 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:41.960706 30791 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:42.004582 30686 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.000708 30587 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.024377 29331 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.048195 30805 raft_consensus.cc:1050] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:42.065029 29177 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.064627 30805 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:42.144696 29458 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.168906 30776 raft_consensus.cc:1050] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:42.178079 29453 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:42.179661 30439 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:42.185354 29572 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:42.189558 30686 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:42.191767 30805 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:42.192302 30776 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:42.211180 30805 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260812 19:06:42.232496 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 82418us
W20260812 19:06:42.286222 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 54386.8us
W20260812 19:06:42.359064 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 70167.4us
W20260812 19:06:42.515379 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 142105us
W20260812 19:06:42.635419 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 208700us
W20260812 19:06:42.643572 30872 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.210s user 0.001s sys 0.001s
I20260812 19:06:42.658849 30805 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:42.707693 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 352178us
I20260812 19:06:42.719306 29458 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:42.753288 30813 raft_consensus.cc:1050] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
W20260812 19:06:42.790148 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 436221us
I20260812 19:06:42.794171 30686 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:42.806732 30813 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:42.827366 30439 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:42.894212 29458 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:42.972546 29176 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:42.992849 30856 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.056s user 0.001s sys 0.000s
I20260812 19:06:42.904353 30686 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.044519 29569 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:43.123335 29575 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:43.125689 29578 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.135169 30684 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.137173 30878 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:43.142464 30906 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260812 19:06:43.150482 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 233758us
I20260812 19:06:43.172997 30681 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:43.185205 29577 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:43.197093 30792 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:43.212260 30805 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260812 19:06:43.228756 29338 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:43.230047 30901 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260812 19:06:43.304591 29338 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260812 19:06:43.321614 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 68274.6us
I20260812 19:06:43.311508 30867 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260812 19:06:43.374742 30878 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:43.387733 30867 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:06:43.388767 30867 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:43.431141 30867 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:43.436309 30685 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.456094 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:06:43.429524 30887 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.444146 30805 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.475033 29575 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.498660 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:06:43.538189 29403 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:06:43.549556 30888 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:06:43.595620 30686 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.611845 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 14e182b70d344161a275142a344cd1ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:43.615625 30901 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.644775 29575 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.656747 29332 raft_consensus.cc:2932] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.664867 29176 catalog_manager.cc:5495] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:06:43.669108 30933 log.cc:924] Time spent T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.070s user 0.000s sys 0.004s
I20260812 19:06:43.688365 29335 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.681732 29453 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.698493 30685 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.699193 29337 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:43.723624 30681 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.726125 29577 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.727025 30805 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:43.732642 29178 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.739636 29178 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:43.731247 30911 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260812 19:06:43.763865 30896 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.771495 30681 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.772810 29575 raft_consensus.cc:2932] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.778940 29437 tablet_service.cc:1514] Processing DeleteTablet for tablet 14e182b70d344161a275142a344cd1ba with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:43.788663 29342 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:43.805279 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet a5c317250947485cab086b4bfab851bd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:43.798748 30681 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.807570 29179 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.809819 30805 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:43.838686 29450 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.837863 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet a5c317250947485cab086b4bfab851bd with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:43.847146 29450 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:43.875583 30908 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260812 19:06:43.884438 30889 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.885215 29577 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:43.887120 30681 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:43.885569 29450 raft_consensus.cc:2932] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:43.942544 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 53758.4us
I20260812 19:06:43.899235 30947 tablet_replica.cc:330] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:43.961572 30886 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260812 19:06:43.973275 30947 raft_consensus.cc:2229] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:43.973984 30947 raft_consensus.cc:2258] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:44.079856 30681 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:44.077942 30947 ts_tablet_manager.cc:1905] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:44.093151 30875 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:44.090705 29177 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:44.118266 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 637fa5c7fa824870a0da61e2bc8400af with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:44.132458 29577 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:44.110107 30792 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:44.151566 30681 raft_consensus.cc:2932] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:44.163434 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 637fa5c7fa824870a0da61e2bc8400af with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:44.181042 29177 catalog_manager.cc:5495] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:44.226270 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2743b6275ccc4fbdbe5f71c667adee8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:44.238087 30951 tablet_replica.cc:330] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:44.238842 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 2743b6275ccc4fbdbe5f71c667adee8b with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:44.240415 30951 raft_consensus.cc:2229] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:44.241745 30951 raft_consensus.cc:2258] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 19:06:44.279888 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 62696.7us
I20260812 19:06:44.284013 30954 tablet_replica.cc:330] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:44.290861 30947 ts_tablet_manager.cc:1918] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:44.300642 30951 ts_tablet_manager.cc:1905] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:44.303602 30947 log.cc:1196] T 14e182b70d344161a275142a344cd1ba P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/14e182b70d344161a275142a344cd1ba
I20260812 19:06:44.308310 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 14e182b70d344161a275142a344cd1ba (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:44.343849 30954 raft_consensus.cc:2229] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:44.346024 30947 tablet_replica.cc:330] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:44.347821 30947 raft_consensus.cc:2229] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:44.349156 30947 raft_consensus.cc:2258] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:44.350593 30954 raft_consensus.cc:2258] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:44.372185 30954 ts_tablet_manager.cc:1905] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:44.374459 30947 ts_tablet_manager.cc:1905] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:44.388886 30951 ts_tablet_manager.cc:1918] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:44.389371 30951 log.cc:1196] T 637fa5c7fa824870a0da61e2bc8400af P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/637fa5c7fa824870a0da61e2bc8400af
I20260812 19:06:44.394045 30947 ts_tablet_manager.cc:1918] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:44.394578 30947 log.cc:1196] T a5c317250947485cab086b4bfab851bd P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/a5c317250947485cab086b4bfab851bd
I20260812 19:06:44.394919 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 637fa5c7fa824870a0da61e2bc8400af (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:44.396052 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet a5c317250947485cab086b4bfab851bd (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:44.423779 30954 ts_tablet_manager.cc:1918] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:44.424259 30954 log.cc:1196] T 2743b6275ccc4fbdbe5f71c667adee8b P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/2743b6275ccc4fbdbe5f71c667adee8b
I20260812 19:06:44.425889 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 2743b6275ccc4fbdbe5f71c667adee8b (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:45.372742 29445 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:45.393096 29575 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.395113 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260812 19:06:45.399840 29399 consensus_peers.cc:487] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d05c973d4068462fa79a7bec57fc6e89. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:45.405823 29339 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.407402 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:45.415819 30958 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.427682 29575 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.437871 29578 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:45.449065 29176 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:45.478636 29335 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:45.489987 29527 consensus_peers.cc:487] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fe3191acdfe44a019be79eda30b9e79b. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:45.492568 29339 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.496093 30884 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:45.495973 29445 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.500630 30884 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:45.518867 30847 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.522187 29335 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.547384 29445 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.565738 29578 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:06:45.581414 29527 consensus_peers.cc:487] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7229b6057eba4aae9e2485a7bf1fcf17. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:45.581403 29335 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.581784 29445 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.585062 30902 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:45.589146 30902 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:45.621982 29177 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.638152 30902 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.643879 29337 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.674376 29458 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:45.675164 29445 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.688515 29178 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.695616 29578 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.702013 30875 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:45.708070 29334 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:45.709623 30875 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260812 19:06:45.713490 29399 consensus_peers.cc:487] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1226d05dc64043078e42cf470348655d. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:45.740665 30958 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.743530 29575 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.746388 29334 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:45.764348 29177 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:45.911409 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 90112us
I20260812 19:06:46.018558 30993 ts_tablet_manager.cc:927] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:46.022536 30993 tablet_copy_client.cc:316] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:46.021749 30992 ts_tablet_manager.cc:927] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:46.034371 30992 tablet_copy_client.cc:316] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:46.037865 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet fe3191acdfe44a019be79eda30b9e79b from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:46.038461 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet fe3191acdfe44a019be79eda30b9e79b from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-fe3191acdfe44a019be79eda30b9e79b
I20260812 19:06:46.065076 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet d05c973d4068462fa79a7bec57fc6e89 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:46.065708 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet d05c973d4068462fa79a7bec57fc6e89 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-d05c973d4068462fa79a7bec57fc6e89
I20260812 19:06:46.077030 29601 tablet_copy_source_session.cc:205] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:46.079610 29474 tablet_copy_source_session.cc:205] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:46.080538 30992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe3191acdfe44a019be79eda30b9e79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:46.088647 30993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d05c973d4068462fa79a7bec57fc6e89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:46.127928 30999 ts_tablet_manager.cc:927] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:46.135134 30998 ts_tablet_manager.cc:927] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:46.150724 30999 tablet_copy_client.cc:316] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:46.153460 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 7229b6057eba4aae9e2485a7bf1fcf17 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:46.154067 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 7229b6057eba4aae9e2485a7bf1fcf17 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-7229b6057eba4aae9e2485a7bf1fcf17
I20260812 19:06:46.154196 30998 tablet_copy_client.cc:316] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:46.157194 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 1226d05dc64043078e42cf470348655d from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:46.157706 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 1226d05dc64043078e42cf470348655d from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-1226d05dc64043078e42cf470348655d
I20260812 19:06:46.170045 29474 tablet_copy_source_session.cc:205] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:46.175490 29601 tablet_copy_source_session.cc:205] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:46.190265 30992 tablet_copy_client.cc:791] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:46.195734 30992 tablet_copy_client.cc:655] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:46.193298 30999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7229b6057eba4aae9e2485a7bf1fcf17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:46.203982 30998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1226d05dc64043078e42cf470348655d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:46.234126 30992 tablet_copy_client.cc:531] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:46.238406 30993 tablet_copy_client.cc:791] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:46.246294 30993 tablet_copy_client.cc:655] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:46.266746 30993 tablet_copy_client.cc:531] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:46.304114 30992 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:46.319561 30999 tablet_copy_client.cc:791] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:46.320703 30999 tablet_copy_client.cc:655] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:46.342691 30999 tablet_copy_client.cc:531] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:46.349728 30993 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:46.357731 30998 tablet_copy_client.cc:791] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:46.358815 30998 tablet_copy_client.cc:655] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:46.366794 30998 tablet_copy_client.cc:531] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:46.405736 30999 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:46.422071 30998 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:46.469102 30993 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:46.470561 30993 tablet_bootstrap.cc:492] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:46.470968 30992 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:46.475471 30992 tablet_bootstrap.cc:492] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:46.477029 30992 ts_tablet_manager.cc:1397] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.173s user 0.029s sys 0.022s
I20260812 19:06:46.474603 30993 ts_tablet_manager.cc:1397] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.125s user 0.005s sys 0.022s
I20260812 19:06:46.486371 30993 raft_consensus.cc:348] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.480952 30992 raft_consensus.cc:348] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.490769 30993 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:46.507501 30993 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.497383 30992 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:46.512768 30992 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.541004 30999 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:46.558615 30999 tablet_bootstrap.cc:492] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:46.571581 30999 ts_tablet_manager.cc:1397] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.166s user 0.031s sys 0.007s
I20260812 19:06:46.573930 30992 ts_tablet_manager.cc:1428] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.097s user 0.009s sys 0.014s
I20260812 19:06:46.574584 30999 raft_consensus.cc:348] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.575587 30999 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:46.575963 30685 raft_consensus.cc:1203] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:46.576614 30999 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.579306 30999 ts_tablet_manager.cc:1428] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:06:46.584035 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-7229b6057eba4aae9e2485a7bf1fcf17 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:46.584463 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-7229b6057eba4aae9e2485a7bf1fcf17 on tablet 7229b6057eba4aae9e2485a7bf1fcf17 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:46.584834 30998 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:46.585753 30998 tablet_bootstrap.cc:492] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:46.593432 30993 ts_tablet_manager.cc:1428] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.110s user 0.006s sys 0.007s
I20260812 19:06:46.601646 30685 raft_consensus.cc:1203] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:46.606935 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-d05c973d4068462fa79a7bec57fc6e89 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:46.607371 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-d05c973d4068462fa79a7bec57fc6e89 on tablet d05c973d4068462fa79a7bec57fc6e89 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:46.593528 30998 ts_tablet_manager.cc:1397] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.172s user 0.027s sys 0.016s
I20260812 19:06:46.626936 30998 raft_consensus.cc:348] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.627835 30998 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:46.628510 30998 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:46.630918 30998 ts_tablet_manager.cc:1428] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260812 19:06:46.634187 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-fe3191acdfe44a019be79eda30b9e79b received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:46.634637 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-fe3191acdfe44a019be79eda30b9e79b on tablet fe3191acdfe44a019be79eda30b9e79b with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:46.655659 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-1226d05dc64043078e42cf470348655d received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:46.656087 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-1226d05dc64043078e42cf470348655d on tablet 1226d05dc64043078e42cf470348655d with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:46.704850 30685 raft_consensus.cc:1203] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:47.036028 31011 raft_consensus.cc:1050] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:47.039197 31011 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.052174 30685 raft_consensus.cc:1203] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:47.056804 30990 raft_consensus.cc:1050] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:47.061866 30990 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.095499 29445 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.096750 30902 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.114063 30685 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.115674 31010 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:47.154798 29342 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.176518 31010 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.210225 29334 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.237305 30685 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.245862 29575 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.275030 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:47.287976 30792 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:47.292614 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.294313 30902 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.319088 29459 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.327838 30685 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.330963 30875 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.339982 29342 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.356236 29575 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.373184 29178 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:47.372668 29177 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.388278 30686 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.408455 30990 raft_consensus.cc:1050] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:47.412884 30990 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:06:47.420416 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 62807.7us
W20260812 19:06:47.423539 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 70507.6us
I20260812 19:06:47.448237 29334 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.454788 29575 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.455016 30686 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.456537 30875 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.463018 29337 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.467542 30990 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:47.468575 31024 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.483335 31027 raft_consensus.cc:1050] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:47.490655 31027 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.507932 30875 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.512629 29575 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.516535 29337 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.518539 29445 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.519915 31030 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.567775 30686 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.591506 30681 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.593086 31030 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.606885 29339 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:47.609083 31030 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:47.636283 29179 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.638666 29575 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.659432 31030 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.666395 29337 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.673313 30686 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.674751 30902 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:47.683753 31032 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.687691 29459 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.687505 30686 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.693429 29458 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.703989 29339 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.705319 30685 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.707693 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:47.712110 29452 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.717865 29569 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.712424 29458 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.705209 30681 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.723237 29179 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.725607 29178 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:47.775333 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 63506.2us
I20260812 19:06:47.809360 30902 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:47.814791 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet fe3191acdfe44a019be79eda30b9e79b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:47.820084 30681 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.846472 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet fe3191acdfe44a019be79eda30b9e79b with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:47.850315 30875 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260812 19:06:47.852447 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 134741us
I20260812 19:06:47.856849 31054 tablet_replica.cc:330] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:47.861068 31054 raft_consensus.cc:2229] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:47.861783 31054 raft_consensus.cc:2258] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:47.868103 31057 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.881178 31054 ts_tablet_manager.cc:1905] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:47.886209 29446 raft_consensus.cc:2932] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.894588 31024 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:47.898864 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet d05c973d4068462fa79a7bec57fc6e89 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:47.897928 31024 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260812 19:06:47.939167 29337 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.954336 29578 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:47.966517 31057 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:47.999258 30681 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:47.999801 31054 ts_tablet_manager.cc:1918] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:48.000514 31054 log.cc:1196] T fe3191acdfe44a019be79eda30b9e79b P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/fe3191acdfe44a019be79eda30b9e79b
I20260812 19:06:48.001461 29342 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:48.004575 31027 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:48.005925 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet fe3191acdfe44a019be79eda30b9e79b (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:48.007786 31027 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260812 19:06:48.017055 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 53474.7us
I20260812 19:06:48.028378 31032 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.034803 30686 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.041558 30681 raft_consensus.cc:2932] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.056089 29578 raft_consensus.cc:2932] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.064124 30685 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.086445 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 7229b6057eba4aae9e2485a7bf1fcf17 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:48.097008 29178 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.107044 29179 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.163060 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 1226d05dc64043078e42cf470348655d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:48.512892 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 1226d05dc64043078e42cf470348655d with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:48.539889 29177 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:48.556543 31067 tablet_replica.cc:330] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:48.558162 31067 raft_consensus.cc:2229] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:48.559090 31067 raft_consensus.cc:2258] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:48.559285 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 7229b6057eba4aae9e2485a7bf1fcf17 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:48.598380 31067 ts_tablet_manager.cc:1905] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:48.598623 31069 tablet_replica.cc:330] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:48.602895 31069 raft_consensus.cc:2229] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:48.603803 31069 raft_consensus.cc:2258] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:48.610986 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet d05c973d4068462fa79a7bec57fc6e89 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:48.637602 31069 ts_tablet_manager.cc:1905] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:48.637033 29332 raft_consensus.cc:2932] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:48.672483 31069 ts_tablet_manager.cc:1918] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:48.673264 31069 log.cc:1196] T 7229b6057eba4aae9e2485a7bf1fcf17 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/7229b6057eba4aae9e2485a7bf1fcf17
I20260812 19:06:48.676100 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 7229b6057eba4aae9e2485a7bf1fcf17 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:48.716205 31071 tablet_replica.cc:330] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:48.737771 31067 ts_tablet_manager.cc:1918] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:48.738353 31067 log.cc:1196] T 1226d05dc64043078e42cf470348655d P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/1226d05dc64043078e42cf470348655d
I20260812 19:06:48.739509 31071 raft_consensus.cc:2229] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:48.740538 31071 raft_consensus.cc:2258] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:48.746740 31071 ts_tablet_manager.cc:1905] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:48.771636 31071 ts_tablet_manager.cc:1918] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:48.772339 31071 log.cc:1196] T d05c973d4068462fa79a7bec57fc6e89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/d05c973d4068462fa79a7bec57fc6e89
I20260812 19:06:48.775511 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet d05c973d4068462fa79a7bec57fc6e89 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:48.791236 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 1226d05dc64043078e42cf470348655d (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
W20260812 19:06:48.800602 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 153493us
W20260812 19:06:49.193337 29374 debug-util.cc:398] Leaking SignalData structure 0x7b0800236b00 after lost signal to thread 27450
W20260812 19:06:49.194453 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08002564e0 after lost signal to thread 29236
W20260812 19:06:49.195317 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08000c27a0 after lost signal to thread 29379
W20260812 19:06:49.196071 29374 debug-util.cc:398] Leaking SignalData structure 0x7b0800236ea0 after lost signal to thread 29496
W20260812 19:06:49.196830 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08002c01c0 after lost signal to thread 29622
W20260812 19:06:49.197573 29374 debug-util.cc:398] Leaking SignalData structure 0x7b08001efc20 after lost signal to thread 30721
I20260812 19:06:49.708292 29339 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:49.714882 29577 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:49.715422 29452 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:06:49.716421 29281 consensus_peers.cc:487] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d3af04408974eeab695361c3b2a8410. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:49.716955 31061 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:49.717842 31074 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:49.726480 31061 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.729370 29452 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.731289 29577 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.741973 29446 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:49.759868 29177 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.762773 29569 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:49.767777 31062 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:49.792146 29399 consensus_peers.cc:487] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 015555682f5f4cf293ca3718be163873. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:49.795080 29339 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:49.800489 30990 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260812 19:06:49.821389 31088 log.cc:924] Time spent T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.067s user 0.001s sys 0.000s
I20260812 19:06:49.835054 31062 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:06:49.848134 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08000f4560 after lost signal to thread 27450
W20260812 19:06:49.849177 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08000fa2a0 after lost signal to thread 29236
W20260812 19:06:49.850314 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800226e80 after lost signal to thread 29379
W20260812 19:06:49.851316 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800318260 after lost signal to thread 29496
W20260812 19:06:49.852304 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800159c60 after lost signal to thread 29622
W20260812 19:06:49.853350 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08002f0c20 after lost signal to thread 30721
I20260812 19:06:49.854951 29342 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.860211 29569 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:49.901327 29446 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:49.918247 29337 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:49.918609 29578 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:49.919536 31062 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:49.920816 29399 consensus_peers.cc:487] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d8ba27ebf754ac098699f3efab22620. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:49.921931 31101 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:50.018555 31101 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.027971 29337 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.028820 29569 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.072943 29179 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.087638 29337 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.103396 29176 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 19:06:50.147994 29281 consensus_peers.cc:487] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf312d0cb2a6414794d20e98e1edf8ba. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:50.151103 29452 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:50.152391 29569 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:50.154007 31061 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:50.157546 31086 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260812 19:06:50.182021 31086 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.185693 29452 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.194336 29578 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.214787 29179 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:50.228891 31123 ts_tablet_manager.cc:927] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:50.241204 31123 tablet_copy_client.cc:316] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:06:50.243295 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 4d3af04408974eeab695361c3b2a8410 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:06:50.243883 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 4d3af04408974eeab695361c3b2a8410 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-4d3af04408974eeab695361c3b2a8410
I20260812 19:06:50.262907 29355 tablet_copy_source_session.cc:205] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:50.266119 31123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d3af04408974eeab695361c3b2a8410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:50.300525 31123 tablet_copy_client.cc:791] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:50.301286 31123 tablet_copy_client.cc:655] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:50.308547 31123 tablet_copy_client.cc:531] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:50.322036 31123 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:50.352736 31126 ts_tablet_manager.cc:927] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:50.398291 31126 tablet_copy_client.cc:316] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:50.402823 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 6d8ba27ebf754ac098699f3efab22620 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:50.403505 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 6d8ba27ebf754ac098699f3efab22620 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-6d8ba27ebf754ac098699f3efab22620
I20260812 19:06:50.405059 31123 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:50.408600 31123 tablet_bootstrap.cc:492] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:50.409449 31123 ts_tablet_manager.cc:1397] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.088s user 0.016s sys 0.027s
I20260812 19:06:50.413525 29474 tablet_copy_source_session.cc:205] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:50.416975 31123 raft_consensus.cc:348] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.420994 31123 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:50.422091 31123 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.428947 31126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d8ba27ebf754ac098699f3efab22620. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:50.439841 31123 ts_tablet_manager.cc:1428] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.030s user 0.013s sys 0.005s
I20260812 19:06:50.466377 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-4d3af04408974eeab695361c3b2a8410 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:06:50.466897 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-4d3af04408974eeab695361c3b2a8410 on tablet 4d3af04408974eeab695361c3b2a8410 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:50.489125 31131 ts_tablet_manager.cc:927] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:50.507025 31131 tablet_copy_client.cc:316] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:50.508487 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 015555682f5f4cf293ca3718be163873 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:50.509070 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 015555682f5f4cf293ca3718be163873 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-015555682f5f4cf293ca3718be163873
I20260812 19:06:50.517378 29474 tablet_copy_source_session.cc:205] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:50.508147 31126 tablet_copy_client.cc:791] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:50.519059 31126 tablet_copy_client.cc:655] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:50.523509 31131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 015555682f5f4cf293ca3718be163873. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:50.528376 31126 tablet_copy_client.cc:531] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:50.547984 31126 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:50.568293 31131 tablet_copy_client.cc:791] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:50.568843 31131 tablet_copy_client.cc:655] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:50.573097 31131 tablet_copy_client.cc:531] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:50.587273 31123 ts_tablet_manager.cc:927] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:50.611018 31123 tablet_copy_client.cc:316] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:06:50.613842 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet bf312d0cb2a6414794d20e98e1edf8ba from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:06:50.614812 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet bf312d0cb2a6414794d20e98e1edf8ba from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-bf312d0cb2a6414794d20e98e1edf8ba
I20260812 19:06:50.618225 31131 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:50.633016 31126 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:50.634102 31126 tablet_bootstrap.cc:492] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:50.634895 31126 ts_tablet_manager.cc:1397] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.087s user 0.018s sys 0.021s
I20260812 19:06:50.643311 29355 tablet_copy_source_session.cc:205] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:50.646085 31126 raft_consensus.cc:348] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.647053 31126 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:50.650622 31123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf312d0cb2a6414794d20e98e1edf8ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:50.654479 31126 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.662057 31126 ts_tablet_manager.cc:1428] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.027s user 0.011s sys 0.000s
I20260812 19:06:50.664744 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-6d8ba27ebf754ac098699f3efab22620 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:50.665510 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-6d8ba27ebf754ac098699f3efab22620 on tablet 6d8ba27ebf754ac098699f3efab22620 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:50.705353 30685 raft_consensus.cc:1203] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:50.710594 31123 tablet_copy_client.cc:791] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:50.711716 31123 tablet_copy_client.cc:655] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:50.735742 31123 tablet_copy_client.cc:531] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:50.746258 31123 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:50.747773 31131 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:50.748705 31131 tablet_bootstrap.cc:492] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:50.749428 31131 ts_tablet_manager.cc:1397] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.140s user 0.031s sys 0.006s
I20260812 19:06:50.752365 31131 raft_consensus.cc:348] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.753240 31131 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:50.753901 31131 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.756320 31131 ts_tablet_manager.cc:1428] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 19:06:50.768654 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-015555682f5f4cf293ca3718be163873 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:50.769117 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-015555682f5f4cf293ca3718be163873 on tablet 015555682f5f4cf293ca3718be163873 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:50.788039 31123 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:50.789033 31123 tablet_bootstrap.cc:492] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:50.789789 31123 ts_tablet_manager.cc:1397] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.044s user 0.021s sys 0.015s
I20260812 19:06:50.792582 31123 raft_consensus.cc:348] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.793537 31123 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:50.794281 31123 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:50.799228 31123 ts_tablet_manager.cc:1428] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20260812 19:06:50.800881 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-bf312d0cb2a6414794d20e98e1edf8ba received from {username='slave'} at 127.0.0.1:41568
I20260812 19:06:50.801314 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-bf312d0cb2a6414794d20e98e1edf8ba on tablet bf312d0cb2a6414794d20e98e1edf8ba with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:50.911697 30685 raft_consensus.cc:1203] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:50.915799 30685 raft_consensus.cc:1203] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:51.070727 30685 raft_consensus.cc:1203] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:51.082227 31083 raft_consensus.cc:1050] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:51.084806 31083 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.144793 30685 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.148098 31086 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.149845 29569 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.154376 31083 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.166935 29452 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260812 19:06:51.193635 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 64752.3us
I20260812 19:06:51.202953 31061 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.214916 31083 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.224023 29458 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.226254 29578 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.244976 29177 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.247010 30685 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.307005 29337 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.313638 30685 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:51.315085 31083 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:51.315500 29446 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:51.322293 31147 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:51.323271 31061 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.330696 30685 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.331888 29452 raft_consensus.cc:2932] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.343556 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 4d3af04408974eeab695361c3b2a8410 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:51.356613 29178 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.374910 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d3af04408974eeab695361c3b2a8410 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:51.389859 31159 tablet_replica.cc:330] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:51.397588 31159 raft_consensus.cc:2229] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:51.398437 31159 raft_consensus.cc:2258] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:51.408406 31159 ts_tablet_manager.cc:1905] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:51.427887 31159 ts_tablet_manager.cc:1918] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:51.428296 31159 log.cc:1196] T 4d3af04408974eeab695361c3b2a8410 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/4d3af04408974eeab695361c3b2a8410
I20260812 19:06:51.429764 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 4d3af04408974eeab695361c3b2a8410 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:51.483407 31099 raft_consensus.cc:1050] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:51.485805 31099 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.493758 29578 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.495306 30685 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.496359 31101 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.498215 31062 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.501032 29339 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.502164 31062 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.529042 31062 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.536499 29337 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.538527 31101 raft_consensus.cc:1050] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:51.543337 31101 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.564077 29178 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.580150 29578 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.580729 30685 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.584605 31162 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.586568 31162 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.591435 29342 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.597265 30990 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.572012 30681 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.609992 31147 raft_consensus.cc:1050] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:51.612668 31147 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.638722 29578 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.664898 29446 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:51.695412 30685 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:51.698422 31099 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:51.700908 29569 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:51.706492 31099 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:51.717433 29569 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.718928 31061 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.718199 31101 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.719983 30990 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.723896 30681 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.726066 31086 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:51.728308 29569 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.732247 30685 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.735641 29577 raft_consensus.cc:2932] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.749845 29458 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:51.752313 31061 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:51.769600 29178 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.781977 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 015555682f5f4cf293ca3718be163873 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:51.793713 31147 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.818946 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 015555682f5f4cf293ca3718be163873 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
W20260812 19:06:51.819551 31172 log.cc:924] Time spent T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.115s user 0.004s sys 0.000s
I20260812 19:06:51.809469 29458 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:51.827919 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 57578us
I20260812 19:06:51.823930 29179 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.821828 29176 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:51.833035 29577 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.861315 31188 tablet_replica.cc:330] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:51.862488 31188 raft_consensus.cc:2229] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:51.863241 31188 raft_consensus.cc:2258] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:51.868002 31188 ts_tablet_manager.cc:1905] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:51.875149 30681 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.973096 29342 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:51.992401 30686 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.025528 31188 ts_tablet_manager.cc:1918] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:52.026239 31188 log.cc:1196] T 015555682f5f4cf293ca3718be163873 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/015555682f5f4cf293ca3718be163873
I20260812 19:06:52.027966 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 015555682f5f4cf293ca3718be163873 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:52.029124 29342 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:52.036015 29575 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:52.037544 31086 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:52.043856 29446 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:52.050900 31061 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.056389 30685 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:52.059219 31192 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:52.059979 29575 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.078411 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bf312d0cb2a6414794d20e98e1edf8ba with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:52.082860 30681 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:52.083750 31201 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.089001 31086 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260812 19:06:52.096025 29334 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:52.100322 31192 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:52.110802 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6d8ba27ebf754ac098699f3efab22620 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:52.110540 30681 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.138604 29334 raft_consensus.cc:2932] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.142880 30685 raft_consensus.cc:2932] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:52.092114 29176 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:52.137007 29178 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:52.173695 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet bf312d0cb2a6414794d20e98e1edf8ba with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:52.235479 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d8ba27ebf754ac098699f3efab22620 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:52.235491 31202 tablet_replica.cc:330] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:52.258046 31202 raft_consensus.cc:2229] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:52.259778 31202 raft_consensus.cc:2258] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:52.265415 31202 ts_tablet_manager.cc:1905] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:52.283463 31203 tablet_replica.cc:330] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:52.289549 31203 raft_consensus.cc:2229] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:52.290361 31203 raft_consensus.cc:2258] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:52.290623 31202 ts_tablet_manager.cc:1918] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:52.291134 31202 log.cc:1196] T bf312d0cb2a6414794d20e98e1edf8ba P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/bf312d0cb2a6414794d20e98e1edf8ba
I20260812 19:06:52.293620 31203 ts_tablet_manager.cc:1905] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:52.293910 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet bf312d0cb2a6414794d20e98e1edf8ba (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:52.311736 31203 ts_tablet_manager.cc:1918] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:52.312261 31203 log.cc:1196] T 6d8ba27ebf754ac098699f3efab22620 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/6d8ba27ebf754ac098699f3efab22620
I20260812 19:06:52.314198 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 6d8ba27ebf754ac098699f3efab22620 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:53.182717 29334 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:06:53.191932 29281 consensus_peers.cc:487] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4737efcfbbb9484781f7335f47b8c2ae. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:53.193459 29577 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.194900 31191 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:53.201476 29446 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.202992 31191 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:53.225373 31195 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.227039 29446 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.242354 29577 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.266974 29575 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:53.274552 29176 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:53.290537 29452 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.291363 29332 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.292026 31145 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:53.296125 31194 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:53.298424 29527 consensus_peers.cc:487] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6469507c6875443ebd9d4c40bb47a54a. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:53.316798 31145 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.318142 29446 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.329819 29575 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:53.341360 29334 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.345410 29446 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.348105 31216 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:53.353184 29527 consensus_peers.cc:487] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79bd0e82dc2545e7867d77ada5cfb445. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:53.357479 29332 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.363185 31190 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:06:53.387411 31190 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.394454 29334 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.391598 29446 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.423622 29446 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:53.425482 29179 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.427167 29179 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.452214 29575 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.454336 31162 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:53.458010 29399 consensus_peers.cc:487] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69bfa848552945a18495c13ca6707d65. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:53.460119 29334 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:53.462046 31232 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:53.469887 31192 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.483819 29575 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.476795 29334 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:53.505738 29179 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:53.744491 31240 ts_tablet_manager.cc:927] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:53.746815 31240 tablet_copy_client.cc:316] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:53.748378 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 6469507c6875443ebd9d4c40bb47a54a from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:53.748895 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 6469507c6875443ebd9d4c40bb47a54a from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-6469507c6875443ebd9d4c40bb47a54a
I20260812 19:06:53.759876 29601 tablet_copy_source_session.cc:205] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:53.763490 31240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6469507c6875443ebd9d4c40bb47a54a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:53.769757 31242 ts_tablet_manager.cc:927] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:06:53.802876 31242 tablet_copy_client.cc:316] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:06:53.805989 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 4737efcfbbb9484781f7335f47b8c2ae from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:06:53.806520 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 4737efcfbbb9484781f7335f47b8c2ae from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-4737efcfbbb9484781f7335f47b8c2ae
I20260812 19:06:53.813236 31240 tablet_copy_client.cc:791] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:53.813879 31240 tablet_copy_client.cc:655] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:53.817755 29355 tablet_copy_source_session.cc:205] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:53.822665 31242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4737efcfbbb9484781f7335f47b8c2ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:53.840116 31240 tablet_copy_client.cc:531] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:53.853878 31240 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:53.863971 31242 tablet_copy_client.cc:791] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:53.865701 31242 tablet_copy_client.cc:655] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:53.870473 31242 tablet_copy_client.cc:531] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:53.880919 31242 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:53.887266 31240 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:53.889447 31240 tablet_bootstrap.cc:492] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:53.893656 31240 ts_tablet_manager.cc:1397] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.040s user 0.023s sys 0.004s
I20260812 19:06:53.896551 31240 raft_consensus.cc:348] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:53.897501 31240 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:53.898322 31240 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:53.916904 31240 ts_tablet_manager.cc:1428] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.023s user 0.010s sys 0.001s
I20260812 19:06:53.926966 31253 ts_tablet_manager.cc:927] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:53.933743 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-6469507c6875443ebd9d4c40bb47a54a received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:53.934315 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-6469507c6875443ebd9d4c40bb47a54a on tablet 6469507c6875443ebd9d4c40bb47a54a with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:53.943184 31240 ts_tablet_manager.cc:927] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:53.950326 31253 tablet_copy_client.cc:316] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:53.952291 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 79bd0e82dc2545e7867d77ada5cfb445 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:53.952356 31240 tablet_copy_client.cc:316] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:53.952864 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 79bd0e82dc2545e7867d77ada5cfb445 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-79bd0e82dc2545e7867d77ada5cfb445
I20260812 19:06:53.955071 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 69bfa848552945a18495c13ca6707d65 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:53.955617 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 69bfa848552945a18495c13ca6707d65 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-69bfa848552945a18495c13ca6707d65
I20260812 19:06:53.974341 29474 tablet_copy_source_session.cc:205] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:53.978770 31240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69bfa848552945a18495c13ca6707d65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:53.979359 29601 tablet_copy_source_session.cc:205] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:53.980940 31242 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:53.984377 31242 tablet_bootstrap.cc:492] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:53.989423 31253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79bd0e82dc2545e7867d77ada5cfb445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:53.990584 31242 ts_tablet_manager.cc:1397] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.110s user 0.018s sys 0.014s
I20260812 19:06:53.999965 31242 raft_consensus.cc:348] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.001343 31240 tablet_copy_client.cc:791] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:54.002293 31242 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:54.002779 31240 tablet_copy_client.cc:655] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:54.003345 31242 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.054224 31242 ts_tablet_manager.cc:1428] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.061s user 0.006s sys 0.001s
I20260812 19:06:54.054454 31253 tablet_copy_client.cc:791] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:54.056223 31253 tablet_copy_client.cc:655] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:54.060947 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-4737efcfbbb9484781f7335f47b8c2ae received from {username='slave'} at 127.0.0.1:41568
I20260812 19:06:54.061333 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-4737efcfbbb9484781f7335f47b8c2ae on tablet 4737efcfbbb9484781f7335f47b8c2ae with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:54.065533 31253 tablet_copy_client.cc:531] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:54.074579 31253 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:54.108294 31240 tablet_copy_client.cc:531] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:54.119647 31240 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:54.131738 31253 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:54.132690 31253 tablet_bootstrap.cc:492] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:54.133436 31253 ts_tablet_manager.cc:1397] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.059s user 0.015s sys 0.011s
I20260812 19:06:54.140309 31253 raft_consensus.cc:348] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.142733 31253 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:54.143407 31253 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.147826 31253 ts_tablet_manager.cc:1428] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.014s user 0.001s sys 0.006s
I20260812 19:06:54.153810 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-79bd0e82dc2545e7867d77ada5cfb445 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:54.154255 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-79bd0e82dc2545e7867d77ada5cfb445 on tablet 79bd0e82dc2545e7867d77ada5cfb445 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:54.177208 31240 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:54.178226 31240 tablet_bootstrap.cc:492] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:54.178939 31240 ts_tablet_manager.cc:1397] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.060s user 0.023s sys 0.015s
I20260812 19:06:54.181285 30685 raft_consensus.cc:1203] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:54.200400 31240 raft_consensus.cc:348] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.201287 31240 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:54.205617 31240 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:54.208202 31240 ts_tablet_manager.cc:1428] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.029s user 0.004s sys 0.002s
I20260812 19:06:54.210011 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-69bfa848552945a18495c13ca6707d65 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:54.210415 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-69bfa848552945a18495c13ca6707d65 on tablet 69bfa848552945a18495c13ca6707d65 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:54.322068 30685 raft_consensus.cc:1203] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:54.347362 30685 raft_consensus.cc:1203] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:54.378621 30685 raft_consensus.cc:1203] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:54.653586 31194 raft_consensus.cc:1050] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:54.656234 31194 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:54.680595 29337 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.682621 29446 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.687459 31190 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260812 19:06:54.691763 30685 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.699920 31270 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260812 19:06:54.722395 31190 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.736105 31216 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.737502 29452 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.746937 30685 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.754683 29337 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.788724 29179 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:54.799377 31270 raft_consensus.cc:1050] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:54.811818 31270 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:54.820312 29337 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.821460 29446 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.838958 30685 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.842767 31270 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.844604 31270 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.848068 31272 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.866371 31261 raft_consensus.cc:1050] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:54.868961 31261 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:54.894191 30681 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.895846 29569 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.898547 31195 raft_consensus.cc:1050] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:54.901265 31195 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:54.898547 29342 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:54.904608 31270 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.907501 30685 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.913568 29342 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:54.916086 31282 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.917002 29575 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:54.918665 31282 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.926224 31282 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:54.985518 29176 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:55.003671 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 54977.6us
I20260812 19:06:55.009822 30681 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.012812 31190 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:55.067555 29332 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.076949 29459 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:55.096660 30681 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:55.102846 31195 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260812 19:06:55.115238 31190 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260812 19:06:55.091030 29452 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.124787 29569 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:55.130915 31303 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.148631 31266 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:55.136793 30685 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.156117 31309 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:55.176215 29577 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:06:55.198740 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 98097.6us
W20260812 19:06:55.199563 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 113729us
I20260812 19:06:55.223824 29178 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.225271 30685 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.227137 31194 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260812 19:06:55.246189 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 43815.9us
I20260812 19:06:55.245656 31272 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.249105 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 6469507c6875443ebd9d4c40bb47a54a with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:55.264983 30685 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.273902 29453 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.276121 30681 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.279053 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6469507c6875443ebd9d4c40bb47a54a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260812 19:06:55.287988 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 54610.3us
I20260812 19:06:55.304885 31331 tablet_replica.cc:330] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:55.308221 31331 raft_consensus.cc:2229] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:55.310710 29575 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.285354 29576 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.317987 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 79bd0e82dc2545e7867d77ada5cfb445 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:55.306241 31266 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.321733 31331 raft_consensus.cc:2258] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:55.340000 31331 ts_tablet_manager.cc:1905] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:55.348088 31194 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:55.325490 31323 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.357640 29177 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:55.352142 30685 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.377269 29452 raft_consensus.cc:2932] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.369047 29177 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.416631 29179 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.428550 31331 ts_tablet_manager.cc:1918] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:55.429155 31331 log.cc:1196] T 6469507c6875443ebd9d4c40bb47a54a P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/6469507c6875443ebd9d4c40bb47a54a
I20260812 19:06:55.435235 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 79bd0e82dc2545e7867d77ada5cfb445 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:55.463297 31334 tablet_replica.cc:330] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:55.462790 29452 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.477751 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 6469507c6875443ebd9d4c40bb47a54a (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:55.463498 29339 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:55.466192 31334 raft_consensus.cc:2229] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:55.482725 31334 raft_consensus.cc:2258] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:55.465739 29342 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.487514 29335 raft_consensus.cc:2932] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.493600 31334 ts_tablet_manager.cc:1905] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:55.495425 29446 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:55.500357 30681 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.501061 29453 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.501727 31323 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:55.506431 31325 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:55.510319 31285 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.516021 30681 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.517377 29342 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.518536 31311 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.519634 31323 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:55.529418 29453 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.547081 29342 raft_consensus.cc:2932] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.556319 30685 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:55.556633 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 4737efcfbbb9484781f7335f47b8c2ae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:55.560633 31334 ts_tablet_manager.cc:1918] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:55.560325 31266 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:55.562762 31334 log.cc:1196] T 79bd0e82dc2545e7867d77ada5cfb445 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/79bd0e82dc2545e7867d77ada5cfb445
I20260812 19:06:55.565778 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 79bd0e82dc2545e7867d77ada5cfb445 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:55.567265 29177 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:55.575131 30685 raft_consensus.cc:2932] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.576470 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 69bfa848552945a18495c13ca6707d65 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:55.596109 29174 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:55.598747 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 4737efcfbbb9484781f7335f47b8c2ae with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:55.607182 31336 tablet_replica.cc:330] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:55.612010 31336 raft_consensus.cc:2229] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:55.613049 31336 raft_consensus.cc:2258] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:55.617313 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 69bfa848552945a18495c13ca6707d65 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:55.624437 31336 ts_tablet_manager.cc:1905] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:55.637923 31336 ts_tablet_manager.cc:1918] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:55.638347 31336 log.cc:1196] T 4737efcfbbb9484781f7335f47b8c2ae P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/4737efcfbbb9484781f7335f47b8c2ae
I20260812 19:06:55.640170 31336 tablet_replica.cc:330] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:55.640693 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 4737efcfbbb9484781f7335f47b8c2ae (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:55.653517 31336 raft_consensus.cc:2229] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:55.654371 31336 raft_consensus.cc:2258] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:55.657581 31336 ts_tablet_manager.cc:1905] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:55.671058 31336 ts_tablet_manager.cc:1918] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:55.671429 31336 log.cc:1196] T 69bfa848552945a18495c13ca6707d65 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/69bfa848552945a18495c13ca6707d65
I20260812 19:06:55.672626 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 69bfa848552945a18495c13ca6707d65 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:56.663637 29575 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:56.669541 29335 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:06:56.670037 29527 consensus_peers.cc:487] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64325c7581194fafac6e6e2b485fbdc2. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:56.670218 29453 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.670962 31194 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.671905 31340 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.681924 31340 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.683539 29453 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.684319 29342 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.704985 29177 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.708114 29575 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:06:56.736658 29527 consensus_peers.cc:487] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0adb15c8e8a8443a9c36474d5714f204. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:56.737627 29342 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.736758 29453 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.739684 31340 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.741420 31304 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.754644 31194 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.756222 29453 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.763499 29335 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.771579 29446 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:56.787528 29335 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.788823 29575 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.785751 29179 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:56.790124 31337 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:56.793243 29399 consensus_peers.cc:487] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf300c57bddf44ffb79e424049fe4c52. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:56.794052 31313 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.809367 31337 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.817408 29342 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.821626 29576 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.822423 29575 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:56.839314 29446 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.841526 29342 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:56.841425 31304 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:56.841856 29174 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.842870 31194 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:06:56.844259 29527 consensus_peers.cc:487] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed9c55b662464bbd8e692e47a2f7cc2c. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:56.859567 31340 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.862222 29335 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.878904 29446 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:56.889760 29174 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:57.165645 31371 ts_tablet_manager.cc:927] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:57.170737 31371 tablet_copy_client.cc:316] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:06:57.172748 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet bf300c57bddf44ffb79e424049fe4c52 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:06:57.173219 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet bf300c57bddf44ffb79e424049fe4c52 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-bf300c57bddf44ffb79e424049fe4c52
I20260812 19:06:57.181159 29474 tablet_copy_source_session.cc:205] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:57.187016 31371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf300c57bddf44ffb79e424049fe4c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:57.201917 31371 tablet_copy_client.cc:791] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:57.202404 31377 ts_tablet_manager.cc:927] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:57.202605 31371 tablet_copy_client.cc:655] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:57.205180 31377 tablet_copy_client.cc:316] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:57.207401 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 64325c7581194fafac6e6e2b485fbdc2 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:57.207973 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 64325c7581194fafac6e6e2b485fbdc2 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-64325c7581194fafac6e6e2b485fbdc2
I20260812 19:06:57.217140 29601 tablet_copy_source_session.cc:205] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:57.218065 31371 tablet_copy_client.cc:531] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:57.222558 31377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64325c7581194fafac6e6e2b485fbdc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:57.228807 31371 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:57.284400 31377 tablet_copy_client.cc:791] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:57.285092 31377 tablet_copy_client.cc:655] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:57.290145 31381 ts_tablet_manager.cc:927] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:57.290585 31377 tablet_copy_client.cc:531] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:57.292039 31371 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:57.293015 31371 tablet_bootstrap.cc:492] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:57.293794 31371 ts_tablet_manager.cc:1397] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.065s user 0.033s sys 0.028s
I20260812 19:06:57.296967 31371 raft_consensus.cc:348] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.298352 31381 tablet_copy_client.cc:316] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:57.297816 31371 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:57.299921 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 0adb15c8e8a8443a9c36474d5714f204 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:57.300454 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 0adb15c8e8a8443a9c36474d5714f204 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-0adb15c8e8a8443a9c36474d5714f204
I20260812 19:06:57.301183 31377 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:57.309275 31371 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.311357 29601 tablet_copy_source_session.cc:205] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:57.316362 31371 ts_tablet_manager.cc:1428] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.022s user 0.007s sys 0.001s
I20260812 19:06:57.319519 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-bf300c57bddf44ffb79e424049fe4c52 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:06:57.320055 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-bf300c57bddf44ffb79e424049fe4c52 on tablet bf300c57bddf44ffb79e424049fe4c52 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:57.327339 31381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0adb15c8e8a8443a9c36474d5714f204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:57.328044 31377 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:57.328948 31377 tablet_bootstrap.cc:492] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:57.329641 31377 ts_tablet_manager.cc:1397] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.029s user 0.027s sys 0.000s
I20260812 19:06:57.335280 31377 raft_consensus.cc:348] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.336161 31377 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:57.336843 31377 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.339344 31377 ts_tablet_manager.cc:1428] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 19:06:57.342873 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-64325c7581194fafac6e6e2b485fbdc2 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:57.344637 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-64325c7581194fafac6e6e2b485fbdc2 on tablet 64325c7581194fafac6e6e2b485fbdc2 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:57.349308 31381 tablet_copy_client.cc:791] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:57.349939 31381 tablet_copy_client.cc:655] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:57.354969 31381 tablet_copy_client.cc:531] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:57.362787 31381 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:57.383773 31381 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:57.384657 31381 tablet_bootstrap.cc:492] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:57.385316 31381 ts_tablet_manager.cc:1397] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.023s user 0.018s sys 0.000s
I20260812 19:06:57.389216 31381 raft_consensus.cc:348] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.389833 31381 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:57.390439 31381 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.392602 31381 ts_tablet_manager.cc:1428] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260812 19:06:57.394690 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-0adb15c8e8a8443a9c36474d5714f204 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:57.395107 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-0adb15c8e8a8443a9c36474d5714f204 on tablet 0adb15c8e8a8443a9c36474d5714f204 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:57.406407 31381 ts_tablet_manager.cc:927] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:57.408330 31381 tablet_copy_client.cc:316] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:06:57.412163 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet ed9c55b662464bbd8e692e47a2f7cc2c from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:06:57.412693 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet ed9c55b662464bbd8e692e47a2f7cc2c from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-ed9c55b662464bbd8e692e47a2f7cc2c
I20260812 19:06:57.421216 29601 tablet_copy_source_session.cc:205] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:06:57.424072 31381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed9c55b662464bbd8e692e47a2f7cc2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:06:57.440764 31381 tablet_copy_client.cc:791] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:06:57.441430 31381 tablet_copy_client.cc:655] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:06:57.446480 31381 tablet_copy_client.cc:531] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:06:57.455725 31381 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:06:57.475651 31381 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:06:57.476466 31381 tablet_bootstrap.cc:492] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:06:57.477013 31381 ts_tablet_manager.cc:1397] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.004s
I20260812 19:06:57.479260 31381 raft_consensus.cc:348] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.479875 31381 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:06:57.480340 31381 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:57.482458 31381 ts_tablet_manager.cc:1428] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260812 19:06:57.484339 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-ed9c55b662464bbd8e692e47a2f7cc2c received from {username='slave'} at 127.0.0.1:46208
I20260812 19:06:57.484813 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-ed9c55b662464bbd8e692e47a2f7cc2c on tablet ed9c55b662464bbd8e692e47a2f7cc2c with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:57.593804 30681 raft_consensus.cc:1203] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:57.753463 30681 raft_consensus.cc:1203] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:57.860139 30681 raft_consensus.cc:1203] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:57.921835 30685 raft_consensus.cc:1203] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:06:58.160097 31323 raft_consensus.cc:1050] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:58.162163 31323 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.167984 29569 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.168745 30685 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.169137 31313 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.171135 31313 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.176963 29335 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.178505 31313 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.187216 31403 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.188877 29342 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.191980 30685 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.208986 29174 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.215871 29576 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.249007 29452 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.254405 30685 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.257457 29335 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.258837 31400 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.260715 31400 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.267241 31313 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.270562 31347 raft_consensus.cc:1050] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:58.269536 30685 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.274113 29342 raft_consensus.cc:2932] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.278656 31347 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.289079 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bf300c57bddf44ffb79e424049fe4c52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:58.291882 29179 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.300568 29342 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.299726 29452 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.302139 30685 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.302102 31340 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:58.304198 31393 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.305133 31395 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.318789 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet bf300c57bddf44ffb79e424049fe4c52 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:06:58.337733 31395 raft_consensus.cc:1050] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:58.340379 31395 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.355865 29342 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.356211 30681 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.358180 31395 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:58.360255 31347 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.371618 29446 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.375053 31340 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:06:58.377453 31428 tablet_replica.cc:330] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:06:58.379510 31428 raft_consensus.cc:2229] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:58.380177 31428 raft_consensus.cc:2258] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:58.381883 31340 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.387488 30685 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.388159 31428 ts_tablet_manager.cc:1905] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:58.395668 31393 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.401486 29342 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.424531 30681 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.421679 29179 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.434078 29177 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:58.439828 31428 ts_tablet_manager.cc:1918] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:58.440300 31428 log.cc:1196] T bf300c57bddf44ffb79e424049fe4c52 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/bf300c57bddf44ffb79e424049fe4c52
I20260812 19:06:58.444537 31438 raft_consensus.cc:1050] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:06:58.446981 31438 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.453473 29453 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:06:58.456067 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55859.5us
I20260812 19:06:58.458294 29445 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.477526 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet bf300c57bddf44ffb79e424049fe4c52 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:58.481722 29342 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.488184 29576 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.504637 29575 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.510973 30685 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.512815 31443 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:06:58.515906 30685 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.516013 29337 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.519104 31443 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260812 19:06:58.523610 30686 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.524874 31340 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.526574 29337 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.528484 31438 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.530680 30686 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.531762 31395 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.538836 31393 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:06:58.541201 31393 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.543761 29337 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.549283 30681 raft_consensus.cc:2932] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.553761 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 64325c7581194fafac6e6e2b485fbdc2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:58.558379 29453 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:06:58.559638 29445 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.564713 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 0adb15c8e8a8443a9c36474d5714f204 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:58.563166 31340 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260812 19:06:58.571290 31443 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.577694 29179 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.586421 29453 raft_consensus.cc:2932] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.597774 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 64325c7581194fafac6e6e2b485fbdc2 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:58.602789 29452 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.600430 31443 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.608989 29174 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.628815 29337 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.633236 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 0adb15c8e8a8443a9c36474d5714f204 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:06:58.622449 30681 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.641600 31451 tablet_replica.cc:330] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:58.643162 31454 tablet_replica.cc:330] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:06:58.651362 31454 raft_consensus.cc:2229] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:58.652238 31454 raft_consensus.cc:2258] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:58.658675 31454 ts_tablet_manager.cc:1905] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:58.647411 31451 raft_consensus.cc:2229] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:58.646678 29174 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.661526 31451 raft_consensus.cc:2258] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:58.678162 31451 ts_tablet_manager.cc:1905] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:58.697431 29569 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:58.711540 31454 ts_tablet_manager.cc:1918] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:58.712075 31454 log.cc:1196] T 0adb15c8e8a8443a9c36474d5714f204 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/0adb15c8e8a8443a9c36474d5714f204
I20260812 19:06:58.713578 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 0adb15c8e8a8443a9c36474d5714f204 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:58.718375 30685 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.721338 29452 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:06:58.722815 31443 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.725046 31438 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:06:58.733592 31451 ts_tablet_manager.cc:1918] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:58.734205 31451 log.cc:1196] T 64325c7581194fafac6e6e2b485fbdc2 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/64325c7581194fafac6e6e2b485fbdc2
I20260812 19:06:58.736230 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 64325c7581194fafac6e6e2b485fbdc2 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:58.733395 31340 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.736901 29445 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.739580 30681 raft_consensus.cc:2932] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.755146 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet ed9c55b662464bbd8e692e47a2f7cc2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:06:58.754441 29176 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:06:58.766973 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet ed9c55b662464bbd8e692e47a2f7cc2c with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:06:58.767937 31451 tablet_replica.cc:330] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:06:58.768996 31451 raft_consensus.cc:2229] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:06:58.769711 31451 raft_consensus.cc:2258] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:06:58.772679 31451 ts_tablet_manager.cc:1905] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:06:58.788374 31451 ts_tablet_manager.cc:1918] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:06:58.788718 31451 log.cc:1196] T ed9c55b662464bbd8e692e47a2f7cc2c P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/ed9c55b662464bbd8e692e47a2f7cc2c
I20260812 19:06:58.790036 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet ed9c55b662464bbd8e692e47a2f7cc2c (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:06:58.868857 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:58.922945 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1226d05dc64043078e42cf470348655d"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:06:58.923545 29452 raft_consensus.cc:595] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:58.925690 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:06:58.926141 29452 raft_consensus.cc:595] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:06:58.928011 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "69bfa848552945a18495c13ca6707d65"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:06:58.928464 29452 raft_consensus.cc:595] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:58.930246 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:06:58.930624 29452 raft_consensus.cc:595] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:58.932543 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:06:58.933045 29452 raft_consensus.cc:595] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:06:58.948760 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:06:58.949381 29342 raft_consensus.cc:595] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:58.969250 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:06:58.969825 29575 raft_consensus.cc:595] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:06:58.971096 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260812 19:06:58.972397 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:06:58.973035 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:06:59.038843 31455 raft_consensus.cc:979] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.039558 31400 raft_consensus.cc:1067] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.041539 30681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:06:59.042084 30681 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.042346 30681 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.046921 30681 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.048736 30681 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:59.050047 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:59.049844 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:59.050725 29342 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.050681 29453 raft_consensus.cc:3032] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.051796 29453 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.052505 29453 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.053547 29453 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.057493 29342 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.059018 29453 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.058854 30629 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:06:59.060349 31383 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.061920 31383 raft_consensus.cc:686] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:06:59.063827 31383 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.075659 29177 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:59.184609 31456 raft_consensus.cc:979] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:06:59.185122 31370 raft_consensus.cc:1067] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:06:59.186333 29453 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:06:59.186769 29453 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.187016 29453 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.192502 29453 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.195103 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:59.195605 29575 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.195750 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:59.196266 29342 raft_consensus.cc:3032] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.196539 29342 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:59.197216 29342 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.198137 29453 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:59.198227 29342 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.201646 29575 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:06:59.202840 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:59.203668 31400 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.204129 31400 raft_consensus.cc:686] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.204365 29342 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:06:59.205036 31400 consensus_queue.cc:237] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.216135 29177 catalog_manager.cc:5495] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:59.236389 31438 raft_consensus.cc:979] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:06:59.236845 31395 raft_consensus.cc:1067] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:06:59.238277 29453 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:06:59.238803 29453 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.239143 29453 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.244868 29453 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.246215 29453 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:59.247009 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:59.247423 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:59.247598 29342 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.247991 29575 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.248270 29575 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:06:59.248914 29575 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.250134 29575 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.254076 29342 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:06:59.255072 29403 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:06:59.256430 29575 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:06:59.258028 31400 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.258526 31400 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.259888 31400 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.267357 31400 raft_consensus.cc:979] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.269804 29177 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:59.272447 31455 raft_consensus.cc:1067] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.274564 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "69bfa848552945a18495c13ca6707d65"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:06:59.275135 30686 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.275507 30686 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.285266 30686 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.286829 30686 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:59.287629 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:59.288280 29342 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.289820 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:59.290449 29453 raft_consensus.cc:3032] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.290727 29453 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.291379 29453 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.292531 29453 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.294817 29342 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.295873 30629 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:06:59.296694 31383 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.297176 31383 raft_consensus.cc:686] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:06:59.298015 31383 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.298529 29453 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.306608 29177 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:59.337905 31455 raft_consensus.cc:979] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.338434 31400 raft_consensus.cc:1067] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.339766 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1226d05dc64043078e42cf470348655d"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:06:59.340246 30686 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.340525 30686 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.345537 30686 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.347395 30686 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:59.348105 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:59.348302 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:59.348709 29453 raft_consensus.cc:3032] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.348912 29569 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.349048 29453 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.349871 29453 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.350867 29453 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.354915 29569 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.355779 30623 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:59.356421 31383 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.356818 31383 raft_consensus.cc:686] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:06:59.357371 29453 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.357601 31383 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.364984 29177 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:59.444634 31455 raft_consensus.cc:979] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:06:59.445168 31474 raft_consensus.cc:1067] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:06:59.446564 29342 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:06:59.447082 29342 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.447418 29342 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.448316 31474 raft_consensus.cc:979] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.448762 31455 raft_consensus.cc:1067] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:06:59.450103 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:06:59.450542 30686 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:06:59.450789 30686 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.452538 29342 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.454269 29342 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:06:59.454689 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:59.454919 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:06:59.455272 29453 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.455569 29569 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.455608 29453 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.456460 29453 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.457363 29453 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.455370 30686 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.460328 30686 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:06:59.461755 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:06:59.462224 29569 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:06:59.462473 29342 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.463639 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:06:59.461740 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:06:59.463505 29453 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:06:59.464329 29445 raft_consensus.cc:3032] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:06:59.464483 31456 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.464653 29445 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:06:59.464975 31456 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:06:59.465351 29445 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.466006 31456 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:06:59.466614 29445 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:06:59.470547 29342 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.471786 30629 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:06:59.472543 31383 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:06:59.473047 31383 raft_consensus.cc:686] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:06:59.473994 31383 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:06:59.479189 29445 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:06:59.482198 29179 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:06:59.482785 29177 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:59.562201 29332 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.563447 31383 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:59.580503 29445 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.581876 31383 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:59.622838 29332 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.624025 31400 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.635005 29578 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.637384 31400 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.724395 29332 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.725525 31400 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.734000 29578 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.737142 31400 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.825762 29453 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.826761 29575 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:06:59.827075 31383 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:59.837720 29332 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.842121 31467 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:59.848204 29445 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.850526 31478 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.855790 29445 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:59.857429 29342 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:59.859668 31395 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.863466 29453 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.861583 31395 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260812 19:06:59.866464 31467 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260812 19:06:59.873534 29527 consensus_peers.cc:487] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c197fd0960e5419fafd6f7eec6f8959a. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:59.875828 29575 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:06:59.877305 31478 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.906832 31395 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:59.920327 29452 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:59.933548 29332 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:06:59.944851 29281 consensus_peers.cc:487] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c0dc7d63c9244e56b10c73fbe2608457. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:06:59.945006 29575 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.945143 29578 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:59.942363 29174 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:06:59.946688 31478 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.949558 29452 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:06:59.952558 31478 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:06:59.954944 29453 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:06:59.955091 31511 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:06:59.952096 29332 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:59.960927 31511 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:06:59.987702 31456 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:59.992671 29578 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:06:59.993196 29452 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.004271 29332 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.012195 31511 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:00.014029 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:00.053579 29452 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.065028 29177 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.084089 29578 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:00.086386 29399 consensus_peers.cc:487] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8ea5787113834ce596fc55df3da0e244. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:00.086526 29332 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:00.087213 31474 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:00.092000 31495 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:00.111809 31495 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.122273 29578 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.135262 29575 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.143692 29176 catalog_manager.cc:5495] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:00.153544 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "015555682f5f4cf293ca3718be163873"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:00.154268 29452 raft_consensus.cc:595] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.156039 29452 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:00.156281 29527 consensus_peers.cc:487] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ebf3eea25534182aa1c9f148898dc8b. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:00.157338 31438 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:00.158505 29332 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:00.159005 29338 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:00.159543 29338 raft_consensus.cc:595] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.159716 29342 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.162489 31395 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:00.170521 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:00.171244 29575 raft_consensus.cc:595] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:00.177084 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:00.177697 29575 raft_consensus.cc:595] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.181476 29445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:00.182090 29445 raft_consensus.cc:595] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:00.183280 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260812 19:07:00.184998 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:00.185611 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:00.186488 31395 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.188275 29452 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.194737 29337 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:00.215687 29179 catalog_manager.cc:5495] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:00.221220 31522 raft_consensus.cc:979] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.225337 31522 raft_consensus.cc:1067] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.229373 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34302
I20260812 19:07:00.238106 30686 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:00.241636 30686 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.249717 30686 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.252704 30686 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.255750 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:00.257449 29578 raft_consensus.cc:3032] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:00.257650 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:00.258288 29578 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:00.258438 29452 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.259166 29578 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.268082 29578 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.270112 31455 raft_consensus.cc:979] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.270619 31455 raft_consensus.cc:1067] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.273352 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "015555682f5f4cf293ca3718be163873"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:07:00.275642 29578 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.276062 30686 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:00.278326 30686 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.278807 30623 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:00.279814 31511 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:00.280280 31511 raft_consensus.cc:686] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:00.281404 29452 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.281051 31511 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.288615 30686 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.291034 30686 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.290627 29174 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:00.292049 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:00.294324 29445 raft_consensus.cc:3032] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:00.294615 29445 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:00.294965 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:00.295281 29445 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.295866 29575 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.296552 29445 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.316000 29575 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.317240 30623 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:00.318123 31511 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:00.318608 31511 raft_consensus.cc:686] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:00.319813 29445 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.319536 31511 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.334048 29174 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:00.345299 31538 ts_tablet_manager.cc:927] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:00.348093 31538 tablet_copy_client.cc:316] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:00.350200 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet c0dc7d63c9244e56b10c73fbe2608457 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:00.350752 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet c0dc7d63c9244e56b10c73fbe2608457 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-c0dc7d63c9244e56b10c73fbe2608457
I20260812 19:07:00.358832 29355 tablet_copy_source_session.cc:205] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:00.361843 31538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0dc7d63c9244e56b10c73fbe2608457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:00.375608 31538 tablet_copy_client.cc:791] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:00.376154 31538 tablet_copy_client.cc:655] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:00.381717 31538 tablet_copy_client.cc:531] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:00.390192 31538 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:00.396157 31370 raft_consensus.cc:979] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.396716 31370 raft_consensus.cc:1067] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:00.398337 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:00.398876 30686 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:00.399204 30686 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.405180 30686 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.410933 30686 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:00.412339 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:00.412904 29342 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:00.413201 29342 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:00.413282 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:00.414211 29452 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.413933 29342 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.417264 29342 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.422744 29452 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.423713 29342 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:00.424170 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:00.426141 31511 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:00.426811 31511 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:00.428388 31511 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:00.443507 31538 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:00.444800 31538 tablet_bootstrap.cc:492] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:00.445770 31538 ts_tablet_manager.cc:1397] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.056s user 0.024s sys 0.010s
I20260812 19:07:00.450757 31538 raft_consensus.cc:348] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.452129 31538 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:00.457696 29174 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:00.453130 31538 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.469002 31544 ts_tablet_manager.cc:927] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.472484 31538 ts_tablet_manager.cc:1428] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.026s user 0.006s sys 0.005s
I20260812 19:07:00.478040 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-c0dc7d63c9244e56b10c73fbe2608457 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:00.478662 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-c0dc7d63c9244e56b10c73fbe2608457 on tablet c0dc7d63c9244e56b10c73fbe2608457 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.502504 31538 ts_tablet_manager.cc:927] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:00.504938 31538 tablet_copy_client.cc:316] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:00.506942 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 8ea5787113834ce596fc55df3da0e244 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:00.506747 31544 tablet_copy_client.cc:316] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:00.507592 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 8ea5787113834ce596fc55df3da0e244 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-8ea5787113834ce596fc55df3da0e244
I20260812 19:07:00.508652 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet c197fd0960e5419fafd6f7eec6f8959a from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:00.509212 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet c197fd0960e5419fafd6f7eec6f8959a from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-c197fd0960e5419fafd6f7eec6f8959a
I20260812 19:07:00.516867 29601 tablet_copy_source_session.cc:205] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:00.519125 29474 tablet_copy_source_session.cc:205] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:00.522509 31544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c197fd0960e5419fafd6f7eec6f8959a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:00.523706 31538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ea5787113834ce596fc55df3da0e244. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:00.539461 31455 raft_consensus.cc:979] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:00.540552 31528 raft_consensus.cc:1067] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:00.544986 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:00.545547 29338 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:00.546685 29338 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.548707 31544 tablet_copy_client.cc:791] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:00.549352 31544 tablet_copy_client.cc:655] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:00.557528 31544 tablet_copy_client.cc:531] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:00.574020 29338 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.577004 29338 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.581660 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:00.582396 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:00.586709 29575 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.586946 29452 raft_consensus.cc:3032] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:00.589383 31544 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:00.589617 29452 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:00.590596 29452 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.591864 29452 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.586588 31538 tablet_copy_client.cc:791] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:00.595631 31538 tablet_copy_client.cc:655] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:00.597781 29575 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:00.598977 29281 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:00.599843 31456 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:00.600509 31456 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:00.601693 31456 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.606365 31538 tablet_copy_client.cc:531] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:00.645848 29452 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:00.669275 31438 raft_consensus.cc:979] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:00.669725 31438 raft_consensus.cc:1067] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:00.678825 29174 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:00.680262 31555 ts_tablet_manager.cc:927] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.679543 29445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:00.680899 29445 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:00.681262 29445 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.681712 31538 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:00.686120 31544 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:00.687983 31544 tablet_bootstrap.cc:492] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:00.688596 31555 tablet_copy_client.cc:316] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:00.688727 31544 ts_tablet_manager.cc:1397] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.100s user 0.020s sys 0.022s
I20260812 19:07:00.690284 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 9ebf3eea25534182aa1c9f148898dc8b from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:00.690789 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 9ebf3eea25534182aa1c9f148898dc8b from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-9ebf3eea25534182aa1c9f148898dc8b
I20260812 19:07:00.695832 29445 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.698468 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:00.698648 29601 tablet_copy_source_session.cc:205] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:00.699029 29338 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.699512 29445 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:00.701902 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:00.702476 29569 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:00.702829 29569 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:00.702512 31544 raft_consensus.cc:348] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.703382 31544 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:00.703565 29569 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.703991 31544 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.704969 29569 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:00.707237 31555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebf3eea25534182aa1c9f148898dc8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:00.708119 31544 ts_tablet_manager.cc:1428] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.019s user 0.005s sys 0.001s
I20260812 19:07:00.712167 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-c197fd0960e5419fafd6f7eec6f8959a received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:00.712555 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-c197fd0960e5419fafd6f7eec6f8959a on tablet c197fd0960e5419fafd6f7eec6f8959a with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.724411 29338 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:00.727079 29403 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:00.727952 31528 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:00.728467 31528 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:00.729012 29569 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:00.729468 31528 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:00.747669 31538 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:00.747380 29174 catalog_manager.cc:5495] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:00.748579 31538 tablet_bootstrap.cc:492] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:00.749291 31538 ts_tablet_manager.cc:1397] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.068s user 0.018s sys 0.007s
I20260812 19:07:00.760768 31538 raft_consensus.cc:348] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.761592 31538 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:00.762176 31538 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.764523 31538 ts_tablet_manager.cc:1428] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260812 19:07:00.767541 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-8ea5787113834ce596fc55df3da0e244 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:00.767922 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-8ea5787113834ce596fc55df3da0e244 on tablet 8ea5787113834ce596fc55df3da0e244 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.771575 29569 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.772794 31511 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:00.774556 31555 tablet_copy_client.cc:791] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:00.784844 31555 tablet_copy_client.cc:655] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:00.790071 29453 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.791458 31511 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:00.792979 31555 tablet_copy_client.cc:531] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:00.808557 31555 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:00.871310 31555 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:00.872303 31555 tablet_bootstrap.cc:492] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:00.872869 31555 ts_tablet_manager.cc:1397] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.065s user 0.025s sys 0.030s
I20260812 19:07:00.875334 31555 raft_consensus.cc:348] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.876200 31555 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:00.877140 31555 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:00.884019 30681 raft_consensus.cc:1203] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:00.887041 31555 ts_tablet_manager.cc:1428] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260812 19:07:00.890214 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-9ebf3eea25534182aa1c9f148898dc8b received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:00.890646 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-9ebf3eea25534182aa1c9f148898dc8b on tablet 9ebf3eea25534182aa1c9f148898dc8b with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:00.891058 29578 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.894137 30681 raft_consensus.cc:1203] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:00.894493 31383 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:00.897907 29453 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.901404 31383 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:00.931689 30681 raft_consensus.cc:1203] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:00.961397 29452 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.964723 31511 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:00.976094 29338 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:00.983681 31572 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:01.044490 29452 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:01.045703 31456 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:01.077031 29569 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:01.083609 31478 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:01.091703 30681 raft_consensus.cc:1203] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:01.168462 29575 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:01.169685 31455 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:01.178884 29338 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:01.180153 31455 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:01.277803 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:01.329593 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:01.330210 29578 raft_consensus.cc:595] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:01.333405 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:01.333902 29575 raft_consensus.cc:595] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:01.340281 29569 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:01.340798 29569 raft_consensus.cc:595] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:01.344134 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:01.344689 29342 raft_consensus.cc:595] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:01.347048 29569 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:01.347582 29569 raft_consensus.cc:595] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:01.348598 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260812 19:07:01.349897 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:01.350399 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:01.373351 31370 raft_consensus.cc:1050] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:01.375860 31370 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:01.392690 30681 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.394296 29445 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.395061 31370 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:01.396221 31456 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.401672 29569 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.403034 31456 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.406015 31370 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.409662 30681 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.417484 29445 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.427002 29179 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:01.430078 31395 raft_consensus.cc:1050] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:01.432520 31395 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:01.445981 29578 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.464594 31607 raft_consensus.cc:979] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:01.465154 31370 raft_consensus.cc:1067] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:01.467085 29445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:01.467839 29445 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:01.468266 29445 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:01.477152 29342 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.479169 29453 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.480684 31395 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.481560 31522 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.482519 30681 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.484081 31522 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.508469 31474 raft_consensus.cc:1050] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:01.512354 31474 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:01.533212 30686 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.533212 29575 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.534662 31455 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.531528 29342 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:01.535861 31528 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.537407 31455 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:01.539319 29445 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:01.546782 29445 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:01.547562 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:01.548283 29569 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:01.553434 31514 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.557544 29569 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:01.565645 29399 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:01.561084 29338 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:01.561837 30685 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.587320 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:01.587929 29337 raft_consensus.cc:3032] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:01.588225 29337 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:01.588102 29445 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.588896 29337 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:01.597689 29337 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:01.604842 30686 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:01.609591 31456 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:01.611963 29337 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:01.630748 31474 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:01.631772 31474 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:01.633225 31474 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:01.666064 31370 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.671952 29177 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.716238 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c0dc7d63c9244e56b10c73fbe2608457 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:01.732324 29176 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:01.773742 30685 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.767257 31455 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:01.839785 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 138144us
I20260812 19:07:01.865487 29576 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260812 19:07:01.884770 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 70370.1us
I20260812 19:07:01.897221 31370 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:01.912264 31635 raft_consensus.cc:1050] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:01.913014 31628 raft_consensus.cc:979] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:01.915061 31628 raft_consensus.cc:1067] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:01.914561 31635 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:01.916802 29576 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.932389 29174 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:01.944839 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 101494us
I20260812 19:07:01.956085 29342 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.960839 31628 raft_consensus.cc:979] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:01.961397 31628 raft_consensus.cc:1067] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:01.962610 31631 raft_consensus.cc:979] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:01.963071 31631 raft_consensus.cc:1067] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:01.967026 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:01.967398 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet c0dc7d63c9244e56b10c73fbe2608457 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:01.967628 29338 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:01.968008 29338 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:01.968099 29575 raft_consensus.cc:2932] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:01.977118 29338 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:01.989256 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46983.4us
W20260812 19:07:01.992918 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 106636us
I20260812 19:07:01.996517 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.006990 29578 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.007300 29578 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:02.008185 30685 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:02.007956 29578 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:01.999115 29338 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:02.008893 29578 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.009055 31522 raft_consensus.cc:979] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:02.009686 31522 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:02.003516 31656 tablet_replica.cc:330] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:02.003952 29342 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:02.010643 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:02.015316 29452 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.019476 29453 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:02.020031 29453 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.020332 29453 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.011552 31522 raft_consensus.cc:1067] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:02.023306 29335 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:02.025364 31648 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:02.030507 29453 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:07:02.033744 29575 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.034817 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:02.035413 29338 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.035802 29338 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.012665 29342 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.038445 29342 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.042284 29338 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.044631 29342 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.046556 29453 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:02.061194 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.061764 29569 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.062090 29569 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:02.012434 31656 raft_consensus.cc:2229] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:02.062822 29569 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.063446 30685 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.063954 29569 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.046867 29177 catalog_manager.cc:5495] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.064348 29337 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.064348 31656 raft_consensus.cc:2258] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:02.065760 31522 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:02.066707 31648 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:02.067663 29342 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:02.062647 31395 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.068929 31656 ts_tablet_manager.cc:1905] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:02.071411 29338 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:02.073607 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:02.074215 30684 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.078305 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:02.078837 30681 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.078060 30686 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.084633 30681 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.085682 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.086366 29567 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.086594 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:02.087713 31608 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.088158 31608 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:02.088975 31608 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.092289 29567 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:02.092007 31648 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.093147 29567 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.094578 29567 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.092152 30684 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.054448 29578 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.056602 29445 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:02.058699 29332 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.059707 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:02.099294 29335 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.098687 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:02.060117 29452 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.102895 31456 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.103391 29282 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:02.103397 31456 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:02.104874 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c197fd0960e5419fafd6f7eec6f8959a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:02.105226 31370 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.105648 31370 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:02.106591 31370 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:02.104871 31456 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.110662 29335 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:02.127831 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.129261 29578 raft_consensus.cc:3032] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.129593 29578 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:02.130342 29578 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:07:02.131613 29578 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.121627 29342 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.139967 29569 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.142657 29567 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:02.142220 29338 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.144517 30686 raft_consensus.cc:2932] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.155117 29403 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:02.156951 31655 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.157438 31655 raft_consensus.cc:686] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:02.158951 31655 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } }
I20260812 19:07:02.170898 29337 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:02.171586 31631 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:02.174515 30685 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.192478 29567 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:02.195914 29578 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:02.200019 31528 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:02.181098 29174 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:02.205775 31656 ts_tablet_manager.cc:1918] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:02.185621 29177 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:02.211953 31656 log.cc:1196] T c0dc7d63c9244e56b10c73fbe2608457 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/c0dc7d63c9244e56b10c73fbe2608457
I20260812 19:07:02.226330 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet c0dc7d63c9244e56b10c73fbe2608457 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:02.227476 29445 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.231662 31528 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:02.247782 29176 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:02.273241 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet c197fd0960e5419fafd6f7eec6f8959a with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:02.278448 31656 tablet_replica.cc:330] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:02.280328 31656 raft_consensus.cc:2229] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:02.281092 31656 raft_consensus.cc:2258] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:02.280972 29179 catalog_manager.cc:5495] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.286747 31656 ts_tablet_manager.cc:1905] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:02.287109 29445 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.295464 30686 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.295560 29342 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.297072 31528 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:02.299932 31655 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:02.304507 29177 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:02.306885 29177 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:02.322583 31656 ts_tablet_manager.cc:1918] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:02.321897 29342 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.323303 31656 log.cc:1196] T c197fd0960e5419fafd6f7eec6f8959a P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/c197fd0960e5419fafd6f7eec6f8959a
I20260812 19:07:02.319133 30686 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.325667 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet c197fd0960e5419fafd6f7eec6f8959a (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:02.317080 31474 raft_consensus.cc:2932] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.345268 29177 catalog_manager.cc:5495] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.348670 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8ea5787113834ce596fc55df3da0e244 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:02.361872 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ea5787113834ce596fc55df3da0e244 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:02.377404 31673 tablet_replica.cc:330] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:02.378659 31673 raft_consensus.cc:2229] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:02.379263 31673 raft_consensus.cc:2258] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:02.382805 31673 ts_tablet_manager.cc:1905] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:02.400302 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:02.405807 31673 ts_tablet_manager.cc:1918] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:02.406656 31673 log.cc:1196] T 8ea5787113834ce596fc55df3da0e244 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/8ea5787113834ce596fc55df3da0e244
I20260812 19:07:02.408902 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 8ea5787113834ce596fc55df3da0e244 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:02.455112 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:02.455637 29452 raft_consensus.cc:595] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:02.457808 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:02.458352 29452 raft_consensus.cc:595] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:02.460734 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:02.461191 29342 raft_consensus.cc:595] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:02.463721 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:02.464218 29342 raft_consensus.cc:595] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:02.467690 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:02.468209 29452 raft_consensus.cc:595] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:02.475217 29576 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:02.475764 29576 raft_consensus.cc:595] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:02.476824 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260812 19:07:02.478008 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:02.478606 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:02.496637 29578 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:02.498152 31608 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:02.511909 30685 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:02.513578 31607 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:02.540555 31648 raft_consensus.cc:979] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:02.541288 31648 raft_consensus.cc:1067] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:02.546402 29452 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:02.546986 29452 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.547325 29452 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.553035 29452 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.555461 29452 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:02.556295 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:02.559576 30685 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.564788 30686 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:02.557718 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.567341 29578 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.567056 31608 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:02.567669 29578 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:02.568418 29578 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.569690 29578 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.576130 30685 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:02.577901 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:02.579177 29342 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:02.579284 29578 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:02.580312 31616 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.580534 31456 raft_consensus.cc:979] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:02.580869 31616 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:02.581017 31456 raft_consensus.cc:1067] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:02.581775 31616 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.586510 31622 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:02.587908 29452 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:02.589793 31608 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:02.595412 30685 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:02.596608 30685 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.597157 30685 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.614456 29578 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:02.608111 30685 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.618386 31608 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:02.643358 30685 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:02.646878 29572 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.648787 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:02.649806 29567 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.651544 29569 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:02.661633 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:02.662192 29337 raft_consensus.cc:3032] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.662472 29337 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:02.671810 29176 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:02.688517 29337 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.652946 31616 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260812 19:07:02.741674 31680 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.103s user 0.001s sys 0.000s
I20260812 19:07:02.748916 29578 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:02.754104 30685 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.759894 31370 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:02.780694 29567 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:02.781132 31630 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:02.783102 31639 raft_consensus.cc:979] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:02.826938 31639 raft_consensus.cc:1067] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:02.834779 30623 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:02.848218 29337 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.844745 31684 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:02.849972 31684 raft_consensus.cc:686] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:02.850935 31684 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.854758 31631 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:02.877852 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47181us
I20260812 19:07:02.886337 29453 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:02.888301 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 9ebf3eea25534182aa1c9f148898dc8b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:02.859007 30684 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.904637 30681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:07:02.905273 30681 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.905625 30681 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.907660 31648 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:02.914512 29337 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:02.919456 31528 raft_consensus.cc:979] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:02.920069 31528 raft_consensus.cc:1067] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:02.928727 31474 raft_consensus.cc:979] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:02.934041 31474 raft_consensus.cc:1067] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:02.947857 29174 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:02.952572 29572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:02.952764 31696 raft_consensus.cc:979] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:02.953699 31695 raft_consensus.cc:1067] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:02.955940 29578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:02.956542 29578 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.956875 29578 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.957350 29452 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:02.957926 29452 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.958308 29452 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.960157 30681 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:02.963644 29453 raft_consensus.cc:2932] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:02.953255 29572 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:02.973588 29572 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:02.967914 29452 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:02.982832 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:02.984510 29337 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:02.984834 29337 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:02.985821 29337 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:02.986892 29337 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.989849 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:02.991922 29452 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:02.990527 29342 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:02.972756 29176 catalog_manager.cc:5495] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:03.006937 29337 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:03.007838 30681 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:03.008172 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:03.009037 31474 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:03.009510 31474 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:03.010356 31474 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.011286 29342 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:03.012585 30629 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:03.014319 31684 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:03.015101 31684 raft_consensus.cc:686] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:03.016479 31684 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:03.018497 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:03.019093 29452 raft_consensus.cc:3032] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:03.019362 29452 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:03.020555 29452 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:03.021944 29452 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.022974 29578 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.030311 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:03.030933 29575 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.032287 29576 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.034050 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:03.034715 29459 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:03.034984 29459 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:03.035684 29459 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.037225 29459 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.038198 31691 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:03.039526 29578 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:03.051714 29452 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:03.058643 30684 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.029609 29572 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.063800 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:03.075933 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:03.070885 29459 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:03.075695 29179 catalog_manager.cc:5495] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:03.073741 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:03.076624 29341 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:03.077343 31691 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:03.082203 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:03.083393 31630 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:03.084388 29452 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:03.084707 29452 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:03.088472 29575 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:03.089617 29572 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:03.092381 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebf3eea25534182aa1c9f148898dc8b with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:03.084172 31630 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:03.098668 29452 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.096813 29342 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.100486 29452 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:03.108502 29452 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:03.109778 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:03.107436 31630 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.111497 31522 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:03.112406 31522 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:03.115110 29341 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:03.126691 29342 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:03.116422 31522 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.171239 29174 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:03.202673 31714 tablet_replica.cc:330] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:03.203853 31714 raft_consensus.cc:2229] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:03.204629 31714 raft_consensus.cc:2258] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:03.275695 29569 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.277184 31714 ts_tablet_manager.cc:1905] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:03.286027 31683 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260812 19:07:03.297385 29179 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 2 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:03.299168 29179 catalog_manager.cc:5495] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:03.326468 31714 ts_tablet_manager.cc:1918] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:03.326993 31714 log.cc:1196] T 9ebf3eea25534182aa1c9f148898dc8b P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/9ebf3eea25534182aa1c9f148898dc8b
I20260812 19:07:03.328691 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 9ebf3eea25534182aa1c9f148898dc8b (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:03.330348 29342 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.332572 31683 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:03.461184 29572 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:03.462320 31639 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:03.481812 29342 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:03.483242 31698 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:03.535632 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:03.563925 29452 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:03.565351 31395 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:03.575268 29342 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:03.577478 31395 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:03.580032 29459 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.580159 29342 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:03.581607 31683 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:03.585270 29572 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:03.585984 29572 raft_consensus.cc:595] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:03.586303 31522 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:03.591879 31522 raft_consensus.cc:979] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:03.592350 31522 raft_consensus.cc:1067] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:03.596865 29453 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:03.598057 31522 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:03.599273 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:03.600030 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:03.601042 29338 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:03.601404 29338 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:03.601492 29452 raft_consensus.cc:595] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:03.607877 29342 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:03.609699 31684 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:03.610090 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:03.610814 29342 raft_consensus.cc:595] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:03.614105 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:03.615790 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:03.616410 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:03.634146 29338 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.637475 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:03.638573 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:03.639151 29572 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:03.647944 29572 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:03.651263 29572 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.652223 29572 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:03.653540 29338 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:03.676070 29572 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:03.677412 29281 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:03.678218 31608 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:03.678675 31608 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:03.679472 31608 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.689167 29176 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 3, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:03.929745 31474 raft_consensus.cc:979] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:03.930253 31639 raft_consensus.cc:1067] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:03.931583 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:03.932009 29338 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:03.932267 29338 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.937676 29338 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.939741 29338 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:03.940032 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:03.940250 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:03.940655 29453 raft_consensus.cc:3032] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:03.940812 29575 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.941000 29453 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:03.941720 29453 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.942917 29453 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:03.947000 29575 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:03.948137 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:03.948814 31607 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:03.949278 31607 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:03.950071 31607 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:03.963737 29176 catalog_manager.cc:5495] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:03.970609 29453 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:04.036010 29453 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.041661 29575 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:04.042866 29338 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260812 19:07:04.043344 29399 consensus_peers.cc:487] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b7629c8adbd447596c2e821c5ac0a99. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:04.043774 31474 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:04.044652 31698 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:04.052532 31639 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.055397 29337 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.055292 29575 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.066411 29338 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.068836 29174 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:04.087467 29281 consensus_peers.cc:487] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97cc899036ba4e0d85662880b5e35132. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:04.087756 29569 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:04.089238 31607 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:04.089906 31688 raft_consensus.cc:979] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:04.090420 29453 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:04.090361 31688 raft_consensus.cc:1067] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:04.094288 29453 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:04.094816 29453 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:04.095160 29453 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:04.097093 31688 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:04.103768 31688 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.106824 29453 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.110903 29453 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:04.111431 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:04.112018 30684 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:04.112409 29569 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.117893 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:04.118671 29337 raft_consensus.cc:3032] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:04.118956 29337 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:04.119477 29572 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.119630 29337 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.124626 29179 catalog_manager.cc:5495] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:04.124527 29453 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:04.121241 30684 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:04.134222 29453 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:04.135818 29399 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:04.136559 31698 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:04.137019 31698 raft_consensus.cc:686] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:04.137854 31698 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.137746 29452 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:04.138839 31607 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:04.138293 29338 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:04.140969 31630 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:07:04.142113 29527 consensus_peers.cc:487] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62647baf8dbe49b9b57e1f296bfa85e6. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:04.142951 29459 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.144588 29337 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:04.145537 31630 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:04.151121 29337 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:04.155416 29179 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:04.171026 31522 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.179904 29338 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.193517 29453 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.196964 29569 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:04.198410 31608 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:04.206804 29459 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.215170 29572 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:04.216693 31639 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260812 19:07:04.217904 29399 consensus_peers.cc:487] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffa1accbf0a749988850cc0f47a8efb0. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:04.221553 29174 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.225786 29338 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:04.228855 31698 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:04.232806 31639 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:07:04.295118 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 54905.6us
I20260812 19:07:04.299103 29572 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.338039 29179 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.373505 29572 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:04.377692 31776 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260812 19:07:04.399538 29338 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:04.462962 31799 ts_tablet_manager.cc:927] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:04.491397 31799 tablet_copy_client.cc:316] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:04.494388 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 8b7629c8adbd447596c2e821c5ac0a99 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:04.494912 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 8b7629c8adbd447596c2e821c5ac0a99 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-8b7629c8adbd447596c2e821c5ac0a99
I20260812 19:07:04.501386 29474 tablet_copy_source_session.cc:205] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:04.507035 31799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b7629c8adbd447596c2e821c5ac0a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.512449 29456 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:04.517321 31608 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:04.524813 31801 ts_tablet_manager.cc:927] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:04.525983 31799 tablet_copy_client.cc:791] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:04.526595 31799 tablet_copy_client.cc:655] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:04.530861 31799 tablet_copy_client.cc:531] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:04.542868 31801 tablet_copy_client.cc:316] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:04.544473 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 97cc899036ba4e0d85662880b5e35132 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:04.544997 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 97cc899036ba4e0d85662880b5e35132 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-97cc899036ba4e0d85662880b5e35132
I20260812 19:07:04.547681 31799 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:04.551350 30684 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:04.552892 31639 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:04.557891 29355 tablet_copy_source_session.cc:205] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:04.563657 31801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97cc899036ba4e0d85662880b5e35132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.584935 29338 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:04.586632 31792 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:04.618883 31807 ts_tablet_manager.cc:927] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:04.633427 31801 tablet_copy_client.cc:791] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:04.634035 31801 tablet_copy_client.cc:655] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:04.643182 31799 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:04.644277 31799 tablet_bootstrap.cc:492] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:04.645105 31799 ts_tablet_manager.cc:1397] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.098s user 0.020s sys 0.019s
I20260812 19:07:04.650918 31801 tablet_copy_client.cc:531] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:04.651716 31799 raft_consensus.cc:348] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.652463 31799 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:04.653141 31799 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.655015 31799 ts_tablet_manager.cc:1428] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260812 19:07:04.656487 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-8b7629c8adbd447596c2e821c5ac0a99 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:04.656916 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-8b7629c8adbd447596c2e821c5ac0a99 on tablet 8b7629c8adbd447596c2e821c5ac0a99 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:04.663946 31807 tablet_copy_client.cc:316] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:04.666045 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 62647baf8dbe49b9b57e1f296bfa85e6 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:04.666581 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 62647baf8dbe49b9b57e1f296bfa85e6 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-62647baf8dbe49b9b57e1f296bfa85e6
I20260812 19:07:04.669561 31801 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:04.673491 29601 tablet_copy_source_session.cc:205] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:04.678057 31807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62647baf8dbe49b9b57e1f296bfa85e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.699738 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:04.710875 31807 tablet_copy_client.cc:791] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:04.712117 31807 tablet_copy_client.cc:655] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:04.719287 31801 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:04.720427 31807 tablet_copy_client.cc:531] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:04.721477 31801 tablet_bootstrap.cc:492] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:04.722642 31801 ts_tablet_manager.cc:1397] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.053s user 0.024s sys 0.011s
I20260812 19:07:04.727589 31801 raft_consensus.cc:348] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.728808 31801 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:04.729940 31801 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.733678 31801 ts_tablet_manager.cc:1428] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260812 19:07:04.736047 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-97cc899036ba4e0d85662880b5e35132 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:04.736588 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-97cc899036ba4e0d85662880b5e35132 on tablet 97cc899036ba4e0d85662880b5e35132 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:04.755980 31807 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:04.764349 29338 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:04.765033 29338 raft_consensus.cc:595] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:04.765933 31801 ts_tablet_manager.cc:927] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:04.767899 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3cc924748e9a482e8536297e097a5226"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:04.768548 29575 raft_consensus.cc:595] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:04.772369 29338 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:04.772912 29338 raft_consensus.cc:595] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:04.778307 29451 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:04.778846 29451 raft_consensus.cc:595] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:04.780964 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260812 19:07:04.782750 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:04.783375 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:04.786505 31801 tablet_copy_client.cc:316] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:04.788304 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet ffa1accbf0a749988850cc0f47a8efb0 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:04.788968 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet ffa1accbf0a749988850cc0f47a8efb0 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-ffa1accbf0a749988850cc0f47a8efb0
I20260812 19:07:04.801548 29474 tablet_copy_source_session.cc:205] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:04.807089 31801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffa1accbf0a749988850cc0f47a8efb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:04.814126 30681 raft_consensus.cc:1203] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20260812 19:07:04.821141 31791 raft_consensus.cc:979] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:04.821655 31791 raft_consensus.cc:1067] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:04.823370 29575 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57270
I20260812 19:07:04.823995 29575 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:04.825371 29575 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:04.848090 31807 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:04.849536 31807 tablet_bootstrap.cc:492] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:04.849221 29575 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.851277 31807 ts_tablet_manager.cc:1397] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.095s user 0.034s sys 0.027s
I20260812 19:07:04.851346 29575 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:04.854693 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:04.855536 29338 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:04.855914 29338 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:04.855789 31807 raft_consensus.cc:348] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.857394 31807 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:04.857467 29338 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.858944 29338 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:04.858654 31807 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:04.861814 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:04.862408 30681 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:04.865260 31807 ts_tablet_manager.cc:1428] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.014s user 0.006s sys 0.005s
I20260812 19:07:04.870627 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-62647baf8dbe49b9b57e1f296bfa85e6 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:04.871204 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-62647baf8dbe49b9b57e1f296bfa85e6 on tablet 62647baf8dbe49b9b57e1f296bfa85e6 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:04.872414 31801 tablet_copy_client.cc:791] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:04.873071 31801 tablet_copy_client.cc:655] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:04.882534 29338 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:04.885388 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:04.888891 31395 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:04.889355 31395 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:04.890429 31801 tablet_copy_client.cc:531] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:04.890123 31395 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:04.904106 30681 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:04.989889 31801 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:05.005697 29176 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:05.052583 31792 raft_consensus.cc:979] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:05.053241 31792 raft_consensus.cc:1067] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:05.067272 29572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:05.067875 29572 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:05.068276 29572 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:05.072738 30681 raft_consensus.cc:1203] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20260812 19:07:05.074147 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 120442us
I20260812 19:07:05.082036 29572 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.084875 29572 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:05.084807 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:05.086382 29341 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:05.086968 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:05.087561 29459 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:05.087849 29459 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:05.088548 29459 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.094323 29459 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:05.104818 29341 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:05.106204 29530 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:05.107719 29459 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:05.107767 31395 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:05.108882 31395 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:05.109549 31395 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.130067 31801 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:05.130761 31801 tablet_bootstrap.cc:492] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:05.131417 31801 ts_tablet_manager.cc:1397] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.142s user 0.028s sys 0.021s
I20260812 19:07:05.133914 31801 raft_consensus.cc:348] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.134778 31801 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:05.135439 31801 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.136737 29176 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:05.141688 31801 ts_tablet_manager.cc:1428] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.010s user 0.001s sys 0.005s
I20260812 19:07:05.144550 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-ffa1accbf0a749988850cc0f47a8efb0 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:05.144979 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-ffa1accbf0a749988850cc0f47a8efb0 on tablet ffa1accbf0a749988850cc0f47a8efb0 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:05.163926 31688 raft_consensus.cc:979] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:05.164510 31779 raft_consensus.cc:1067] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:05.166115 29451 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:05.166640 29451 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:05.166982 29451 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.172482 29451 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.174770 29451 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:05.223107 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:05.225557 29572 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.229295 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:05.230119 29337 raft_consensus.cc:3032] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:05.230607 29337 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:05.232618 29337 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.237983 29337 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.251293 30684 raft_consensus.cc:1203] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:05.253233 29337 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:05.258294 31395 raft_consensus.cc:979] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:05.259438 31630 raft_consensus.cc:1067] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:05.262691 29403 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:05.267678 29572 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:05.272600 31792 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:05.273293 31792 raft_consensus.cc:686] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:05.275143 31792 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:05.295837 29341 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3cc924748e9a482e8536297e097a5226"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:05.296746 29341 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:05.297248 29341 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.314179 30684 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.309247 29341 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:05.319562 29341 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:05.320859 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:05.321575 29575 raft_consensus.cc:3032] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:05.321856 29575 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:05.322405 31788 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:05.322819 29575 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:05.324342 29575 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.336544 31639 raft_consensus.cc:1050] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:05.338929 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:05.346290 29451 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:05.369468 31639 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.392128 29575 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:05.393778 29281 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:05.402843 31608 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:05.403461 31608 raft_consensus.cc:686] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:05.404431 31608 consensus_queue.cc:237] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:05.417245 30685 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.418870 31639 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:05.432265 29572 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.437348 29337 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.438059 31792 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:05.442631 31639 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:05.445019 29451 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:05.454764 29341 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.465124 31630 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:05.467456 31639 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.468107 29572 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.470826 29179 catalog_manager.cc:5495] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:05.461828 29176 catalog_manager.cc:5495] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:05.492063 29342 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.494923 30684 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.513059 29179 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.520893 30685 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Advancing to term 3
I20260812 19:07:05.536880 29459 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.545089 29572 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.545236 30685 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:05.546849 31792 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:05.550464 30684 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.551846 31792 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:05.555307 31522 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:05.568006 31522 raft_consensus.cc:1050] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
W20260812 19:07:05.568444 31522 raft_consensus.cc:1054] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Unable to promote non-voter a34c47dd29c14748a018d716ac6c39df: Illegal state: Leader has not yet committed an operation in its own term
I20260812 19:07:05.573694 29459 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:05.573513 31795 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.577916 31522 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:05.582465 30685 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.579886 29575 raft_consensus.cc:2932] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.594337 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8b7629c8adbd447596c2e821c5ac0a99 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:05.609401 29174 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.621584 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b7629c8adbd447596c2e821c5ac0a99 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:05.626891 29341 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:05.628579 31522 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:05.653396 31630 raft_consensus.cc:1050] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:05.655851 31630 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.663466 30684 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.665486 31522 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:05.669129 29451 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.672628 31395 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:05.674691 31395 raft_consensus.cc:1050] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:05.677306 31395 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.681318 29342 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:05.682770 29572 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:05.683956 31522 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:05.686386 31850 tablet_replica.cc:330] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:05.687664 31850 raft_consensus.cc:2229] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:05.688383 31850 raft_consensus.cc:2258] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:05.690346 31639 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:05.691342 30684 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Advancing to term 2
I20260812 19:07:05.691933 31850 ts_tablet_manager.cc:1905] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:05.699297 31788 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.703795 30685 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:05.729130 30684 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:05.731683 29459 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.708722 31630 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:05.730468 29451 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:05.730161 29341 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:05.737749 29341 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.741621 31639 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260812 19:07:05.744897 31395 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:05.748092 31522 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:05.750036 31850 ts_tablet_manager.cc:1918] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260812 19:07:05.750638 31850 log.cc:1196] T 8b7629c8adbd447596c2e821c5ac0a99 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/8b7629c8adbd447596c2e821c5ac0a99
I20260812 19:07:05.752766 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 8b7629c8adbd447596c2e821c5ac0a99 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:05.765612 30681 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.781411 29176 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.786238 31639 raft_consensus.cc:1050] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
W20260812 19:07:05.786811 31639 raft_consensus.cc:1054] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Unable to promote non-voter a34c47dd29c14748a018d716ac6c39df: Illegal state: Leader has not yet committed an operation in its own term
W20260812 19:07:05.805114 31860 log.cc:924] Time spent T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.073s user 0.003s sys 0.000s
I20260812 19:07:05.809767 29342 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:05.812074 31792 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:05.821908 31630 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.827862 31781 raft_consensus.cc:1050] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:05.830394 31781 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.831225 30681 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.834002 29459 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.835996 29575 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:05.846680 29569 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:05.846868 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:05.853621 30685 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.855026 31395 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:05.865049 31795 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:05.867018 30685 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.865602 31522 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.846668 30684 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:05.890226 29451 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:05.863700 29337 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.895748 31788 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:05.912225 29174 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.947991 31792 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:05.950958 29450 raft_consensus.cc:2932] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:05.976516 29451 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:06.015236 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet ffa1accbf0a749988850cc0f47a8efb0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:06.027640 29575 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.057564 30681 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:06.066672 31788 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.012s
I20260812 19:07:06.080677 29341 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:06.080504 31877 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:06.095012 29569 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:06.097363 31873 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260812 19:07:06.103034 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 54777us
I20260812 19:07:06.108549 29174 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 5 to 6, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.129046 29342 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:06.130240 30685 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:06.133131 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 43771.7us
I20260812 19:07:06.132580 31630 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.151691 31395 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:06.184940 31885 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260812 19:07:06.186061 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet ffa1accbf0a749988850cc0f47a8efb0 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:06.187793 31792 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.190629 31850 tablet_replica.cc:330] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:06.191931 31850 raft_consensus.cc:2229] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:06.192888 31850 raft_consensus.cc:2258] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:06.204080 30685 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.210673 29178 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.223848 31850 ts_tablet_manager.cc:1905] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:06.265610 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 62647baf8dbe49b9b57e1f296bfa85e6 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
W20260812 19:07:06.274226 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 85842.9us
I20260812 19:07:06.280174 29335 raft_consensus.cc:2932] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:06.286139 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49257.2us
I20260812 19:07:06.321779 29569 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.325173 29178 catalog_manager.cc:5495] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.339771 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 62647baf8dbe49b9b57e1f296bfa85e6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:06.408658 31897 tablet_replica.cc:330] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:06.410013 31897 raft_consensus.cc:2229] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:06.410811 31897 raft_consensus.cc:2258] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:06.415551 29341 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:06.418108 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 58680.8us
I20260812 19:07:06.421561 31897 ts_tablet_manager.cc:1905] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:06.430970 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:06.431675 29575 raft_consensus.cc:595] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:06.437001 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "59710c65553f42188b78bc9bc0324542"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:06.437625 29450 raft_consensus.cc:595] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:06.436949 31630 raft_consensus.cc:979] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:06.438820 31788 raft_consensus.cc:1067] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:06.440610 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:06.441311 29575 raft_consensus.cc:595] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:06.445230 29335 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:06.445812 29335 raft_consensus.cc:595] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:06.447193 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260812 19:07:06.449689 29456 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:06.451723 29456 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:06.452077 29456 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.441457 31850 ts_tablet_manager.cc:1918] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260812 19:07:06.453930 31850 log.cc:1196] T ffa1accbf0a749988850cc0f47a8efb0 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/ffa1accbf0a749988850cc0f47a8efb0
I20260812 19:07:06.455470 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet ffa1accbf0a749988850cc0f47a8efb0 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:06.459774 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:06.460493 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:06.461217 29456 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.472684 31395 raft_consensus.cc:979] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:06.473176 31395 raft_consensus.cc:1067] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:06.473409 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:06.474025 29337 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.474303 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:06.474889 29569 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:06.475165 29569 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:06.475872 29569 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.476701 29569 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.482163 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:06.482724 29335 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:06.483045 29335 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.488543 29456 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:06.491103 29337 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:06.493256 29403 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:06.494179 31781 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:06.494647 29569 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:06.494726 31781 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:06.495818 31781 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.503059 31897 ts_tablet_manager.cc:1918] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:06.503548 31897 log.cc:1196] T 62647baf8dbe49b9b57e1f296bfa85e6 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/62647baf8dbe49b9b57e1f296bfa85e6
I20260812 19:07:06.508805 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 62647baf8dbe49b9b57e1f296bfa85e6 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:06.513803 29335 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.515750 29335 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:06.517040 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:06.517577 29569 raft_consensus.cc:3032] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:06.517838 29569 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:06.518705 29569 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.519804 29569 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.521479 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:06.524652 30681 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.526787 29569 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:06.527792 29281 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:06.530164 31878 raft_consensus.cc:2781] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:06.530635 31878 raft_consensus.cc:686] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:06.531476 31878 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.536895 30681 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:06.559995 29178 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:06.564991 29174 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:06.570770 31878 raft_consensus.cc:979] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:06.571305 31878 raft_consensus.cc:1067] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:06.577508 29569 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57270
I20260812 19:07:06.578249 29569 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:06.578578 29569 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:06.586493 29569 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.588073 29569 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:06.588786 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:06.589363 29335 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:06.589407 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:06.589701 29335 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:06.590041 30681 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:06.590454 29335 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.591684 29335 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:06.600219 30681 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:06.600210 29335 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:06.601600 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:06.602537 31788 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:06.602993 31788 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:06.603838 31788 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.615413 29178 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:06.684959 31795 raft_consensus.cc:979] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:06.685422 31795 raft_consensus.cc:1067] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:06.686782 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "59710c65553f42188b78bc9bc0324542"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:06.687242 29335 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:06.687516 29335 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.693425 29335 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.695559 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:06.695968 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:06.696439 29335 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:06.696561 29569 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.696096 29456 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:06.698354 29456 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:06.699160 29456 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.700173 29456 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:06.702119 29569 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:06.703805 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:06.704512 31879 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:06.704878 31879 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:06.705766 29456 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:06.705629 31879 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:06.715445 29178 catalog_manager.cc:5495] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:06.740615 29451 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:06.746579 30681 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:06.746708 29335 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:06.748333 31781 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:06.752071 31787 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:06.759019 31795 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.765200 30681 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.767858 29341 raft_consensus.cc:2932] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.769688 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 97cc899036ba4e0d85662880b5e35132 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:06.777760 29178 catalog_manager.cc:5495] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:06.792903 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 97cc899036ba4e0d85662880b5e35132 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:06.795311 31915 tablet_replica.cc:330] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:06.796191 31915 raft_consensus.cc:2229] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:06.796811 31915 raft_consensus.cc:2258] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:06.800153 31915 ts_tablet_manager.cc:1905] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:06.812772 31915 ts_tablet_manager.cc:1918] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260812 19:07:06.813109 31915 log.cc:1196] T 97cc899036ba4e0d85662880b5e35132 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/97cc899036ba4e0d85662880b5e35132
I20260812 19:07:06.814615 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 97cc899036ba4e0d85662880b5e35132 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:06.884380 29335 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:06.885699 31795 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:06.896592 29569 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:06.898011 31781 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:06.968078 30681 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:06.969194 31877 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:06.978035 29569 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:06.986143 31779 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:07.024801 29341 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:07.025802 31788 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:07.034884 30681 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:07.037436 31788 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:07.226716 29451 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:07.227942 31879 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:07.237336 29569 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:07.238590 31879 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:07.578660 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:07.614317 29575 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:07.614835 29575 raft_consensus.cc:595] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:07.617125 29341 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:07.617583 29341 raft_consensus.cc:595] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:07.620170 29335 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a5c317250947485cab086b4bfab851bd"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:07.620754 29335 raft_consensus.cc:595] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:07.623194 29456 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:07.623739 29456 raft_consensus.cc:595] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:07.624858 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260812 19:07:07.626555 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:07.627210 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:07.813697 31926 raft_consensus.cc:979] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:07.814174 31630 raft_consensus.cc:1067] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:07.815632 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "14e182b70d344161a275142a344cd1ba"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:07.816140 29335 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:07.816468 29335 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.821422 29335 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:07.823844 29335 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:07.824431 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:07.824934 30685 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.827097 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:07.827685 29569 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:07.827950 29569 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:07.828637 29569 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:07.830677 30685 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:07.831200 29569 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.831813 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:07.832516 31879 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:07.833688 31879 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:07.834534 31879 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:07.839970 29569 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:07.855499 31877 raft_consensus.cc:979] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:07.855986 31877 raft_consensus.cc:1067] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:07.856307 29174 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:07.866948 29451 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:07.867491 29451 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:07.867846 29451 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.869779 29335 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:07.882620 29451 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:07.884832 29451 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:07.884959 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:07.885643 29337 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:07.885922 29337 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:07.886668 29337 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:07.887867 29337 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.885020 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:07.890673 29569 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:07.896051 29337 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:07.900293 29451 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:07.901432 29569 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:07.902535 31879 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:07.903532 29403 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:07.904371 31781 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:07.904814 31781 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:07.905660 31781 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:07.916235 29569 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:07.917853 31879 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:07.919394 31779 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:07:07.923892 29281 consensus_peers.cc:487] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c89f8a2f65164cd08d1e760f9005e496. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:07.923058 29451 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:07.950733 29569 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:07.952206 29337 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:07.959017 29174 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:07.966500 31879 raft_consensus.cc:979] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:07.966974 31879 raft_consensus.cc:1067] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:07.964200 29176 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:07.969326 30685 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5c317250947485cab086b4bfab851bd"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:07.969758 30685 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:07.970127 30685 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:07.974480 29281 consensus_peers.cc:487] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: db322776a340462b91983d8e448cfe39. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:07.977579 29451 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:07.978586 29569 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:07.979501 31933 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:07.981043 31877 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:07.985615 30685 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:07.988489 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:07.989108 29572 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:07.993366 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:07.993978 29335 raft_consensus.cc:3032] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:07.994251 29335 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:07.994948 29335 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:07.996054 29335 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:08.010375 29572 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:08.018980 30623 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:08.021001 29335 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:08.026317 30685 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:08.031721 31953 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:08.036134 31953 raft_consensus.cc:686] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:08.039182 31953 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:08.190397 31779 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.207057 29572 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.209233 31781 raft_consensus.cc:979] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:08.208376 29176 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:08.212000 31795 raft_consensus.cc:1067] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:08.221310 29174 catalog_manager.cc:5495] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:08.235285 29569 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:08.235889 29569 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:08.236214 29569 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:08.236121 29450 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.244298 29569 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:08.244232 29451 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.247084 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:08.247188 29569 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:08.247653 29450 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:08.249333 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:08.249877 30681 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:08.263634 30681 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:08.247951 29450 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:08.271399 29527 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:08.272130 31630 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:08.272631 31630 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:07:08.274452 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 43944.1us
I20260812 19:07:08.276101 29569 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:08.273427 31630 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:08.281544 29450 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:08.283592 29450 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:08.284888 31933 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:08.287034 29342 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:08.289829 31795 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:07:08.291985 29399 consensus_peers.cc:487] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 647dfc8bad934787a924a06988dda474. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:08.298928 29572 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:08.300364 31935 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:08.309064 29450 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:08.318581 30681 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:08.322695 31779 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:08.331367 29176 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:08.334213 31781 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.337062 29578 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.341343 29342 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.355799 29176 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.357542 29572 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:07:08.359277 31795 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:08.367744 29337 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.399667 29572 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:08.400708 29281 consensus_peers.cc:487] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e87477250ac44a318f7c0aef2d89fa36. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:08.401859 29451 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:08.402863 31779 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:08.403730 31933 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:08.428712 29578 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.435581 29451 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.438330 29337 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:07:08.441087 31795 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:08.423529 31933 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.450353 29341 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:08.452178 31954 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:08.461364 31991 ts_tablet_manager.cc:927] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:08.464258 29176 catalog_manager.cc:5495] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:08.489964 29569 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:08.491439 31825 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260812 19:07:08.500388 31991 tablet_copy_client.cc:316] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:08.505352 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet c89f8a2f65164cd08d1e760f9005e496 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:08.505978 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet c89f8a2f65164cd08d1e760f9005e496 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-c89f8a2f65164cd08d1e760f9005e496
I20260812 19:07:08.514705 29355 tablet_copy_source_session.cc:205] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:08.518505 31991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c89f8a2f65164cd08d1e760f9005e496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:08.528585 31999 ts_tablet_manager.cc:927] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:08.534631 31999 tablet_copy_client.cc:316] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:08.536055 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet db322776a340462b91983d8e448cfe39 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:08.536475 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet db322776a340462b91983d8e448cfe39 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-db322776a340462b91983d8e448cfe39
I20260812 19:07:08.539853 31991 tablet_copy_client.cc:791] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:08.540441 31991 tablet_copy_client.cc:655] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:08.542881 29355 tablet_copy_source_session.cc:205] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:08.544813 31991 tablet_copy_client.cc:531] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:08.546195 31999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db322776a340462b91983d8e448cfe39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:08.557128 31991 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:08.563208 31999 tablet_copy_client.cc:791] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:08.563808 31999 tablet_copy_client.cc:655] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:08.582592 31999 tablet_copy_client.cc:531] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:08.587330 31991 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:08.588475 31991 tablet_bootstrap.cc:492] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:08.589226 31991 ts_tablet_manager.cc:1397] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.032s user 0.016s sys 0.008s
I20260812 19:07:08.592186 31991 raft_consensus.cc:348] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.593104 31991 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:08.593722 31991 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.594651 31999 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:08.596323 31991 ts_tablet_manager.cc:1428] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260812 19:07:08.599844 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-c89f8a2f65164cd08d1e760f9005e496 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:08.600222 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-c89f8a2f65164cd08d1e760f9005e496 on tablet c89f8a2f65164cd08d1e760f9005e496 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:08.620522 31999 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:08.621225 31999 tablet_bootstrap.cc:492] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:08.621748 31999 ts_tablet_manager.cc:1397] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.027s user 0.016s sys 0.001s
I20260812 19:07:08.624068 31999 raft_consensus.cc:348] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.624644 31999 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:08.625195 31999 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.627507 31999 ts_tablet_manager.cc:1428] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 19:07:08.629555 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-db322776a340462b91983d8e448cfe39 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:08.629897 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-db322776a340462b91983d8e448cfe39 on tablet db322776a340462b91983d8e448cfe39 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:08.660986 29450 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:08.661214 30685 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:08.662299 31630 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:08.662982 31960 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:08.670739 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:08.770473 29335 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:08.771031 29335 raft_consensus.cc:595] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:08.773411 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:08.774039 29450 raft_consensus.cc:595] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:08.781126 29572 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:08.781683 29572 raft_consensus.cc:595] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:08.785143 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:08.786885 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:08.787489 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:08.841428 31999 ts_tablet_manager.cc:927] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:08.845280 31999 tablet_copy_client.cc:316] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:08.846612 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet e87477250ac44a318f7c0aef2d89fa36 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:08.847126 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet e87477250ac44a318f7c0aef2d89fa36 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-e87477250ac44a318f7c0aef2d89fa36
I20260812 19:07:08.858794 29355 tablet_copy_source_session.cc:205] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:08.867586 31999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87477250ac44a318f7c0aef2d89fa36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:08.880795 31999 tablet_copy_client.cc:791] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:08.882205 31999 tablet_copy_client.cc:655] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:08.885205 31999 tablet_copy_client.cc:531] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:08.893208 31999 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:08.898494 31991 ts_tablet_manager.cc:927] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:08.913682 30681 raft_consensus.cc:1203] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:08.917658 31991 tablet_copy_client.cc:316] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:08.923152 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 647dfc8bad934787a924a06988dda474 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:08.923700 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 647dfc8bad934787a924a06988dda474 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-647dfc8bad934787a924a06988dda474
I20260812 19:07:08.930977 31999 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:08.931490 29474 tablet_copy_source_session.cc:205] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:08.931862 31999 tablet_bootstrap.cc:492] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:08.932602 31999 ts_tablet_manager.cc:1397] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.040s user 0.014s sys 0.008s
I20260812 19:07:08.934703 31991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 647dfc8bad934787a924a06988dda474. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:08.935317 31999 raft_consensus.cc:348] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.936179 31999 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:08.936800 31999 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:08.944150 31999 ts_tablet_manager.cc:1428] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260812 19:07:08.945672 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-e87477250ac44a318f7c0aef2d89fa36 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:08.946080 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-e87477250ac44a318f7c0aef2d89fa36 on tablet e87477250ac44a318f7c0aef2d89fa36 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:08.956125 31991 tablet_copy_client.cc:791] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:08.956744 31991 tablet_copy_client.cc:655] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:08.962445 31991 tablet_copy_client.cc:531] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:08.973434 31991 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:08.997491 31991 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:08.998536 31991 tablet_bootstrap.cc:492] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:08.999266 31991 ts_tablet_manager.cc:1397] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.002s
I20260812 19:07:09.001938 31991 raft_consensus.cc:348] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:09.002796 31991 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:09.003439 31991 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:09.005788 31991 ts_tablet_manager.cc:1428] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260812 19:07:09.007829 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-647dfc8bad934787a924a06988dda474 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:09.008162 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-647dfc8bad934787a924a06988dda474 on tablet 647dfc8bad934787a924a06988dda474 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:09.018857 30681 raft_consensus.cc:1203] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:09.136034 31630 raft_consensus.cc:979] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:09.136574 31926 raft_consensus.cc:1067] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:09.138051 29341 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:09.138502 29341 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:09.138784 29341 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:09.145247 29341 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.147581 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:09.148038 29341 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:09.148245 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:09.148846 30681 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:09.148134 29572 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:09.150239 29572 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:09.150939 29572 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.152127 29572 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:09.155119 30681 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:09.156332 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:09.157282 31933 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:09.157773 31933 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:09.158820 29572 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:09.158699 31933 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.170024 31879 raft_consensus.cc:979] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:09.170574 31879 raft_consensus.cc:1067] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:09.172271 29451 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:09.172879 29451 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:09.173216 29451 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:09.174770 29174 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:09.180774 29451 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.183411 29451 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:09.183804 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:09.184487 29572 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:09.187135 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:09.187638 29335 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:09.187853 29335 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:09.188537 29335 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.189590 29335 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:09.195767 29572 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:09.195943 29335 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:09.198228 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:09.199185 31976 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:09.199632 31976 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:09.200270 31976 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.210969 29174 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:09.223013 30685 raft_consensus.cc:1203] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:09.298658 31795 raft_consensus.cc:979] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:09.299153 31976 raft_consensus.cc:1067] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:09.300712 29572 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:09.301242 29572 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:09.301563 29572 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:09.310439 29572 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.312636 30685 raft_consensus.cc:1203] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:09.313457 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:09.314110 29335 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:09.316375 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:09.316991 29450 raft_consensus.cc:3032] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:09.317288 29450 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:09.317989 29450 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.318969 29450 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:09.324127 29572 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:09.330394 29335 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:09.331419 29530 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:09.332644 29450 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:09.333869 31926 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:09.335980 31926 raft_consensus.cc:686] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:09.336875 31926 consensus_queue.cc:237] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:09.348296 29179 catalog_manager.cc:5495] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 2 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:09.524344 31779 raft_consensus.cc:1050] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:09.526325 31779 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.531847 29578 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.532155 30685 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.532198 29451 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.533469 31779 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.534456 31933 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.535276 31879 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.555912 31933 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.557555 30681 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.558223 29450 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.584748 29179 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.590891 29578 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.602403 29342 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:09.609242 31935 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.612463 29572 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:09.613807 31976 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.616959 29337 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.622394 30685 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:09.626118 31779 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:09.628660 29450 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:09.633826 31879 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:09.637050 31877 raft_consensus.cc:1050] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:09.639513 31877 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.639611 31933 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.641525 30681 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.650743 29451 raft_consensus.cc:2932] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.662549 31779 raft_consensus.cc:1050] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:09.665078 31779 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.671173 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c89f8a2f65164cd08d1e760f9005e496 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:09.689251 30685 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.689574 29578 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.690155 29450 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.691424 31879 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.693423 31879 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.696301 31779 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:09.710924 29459 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.714778 31877 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:09.719256 29179 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:09.739462 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 48066.9us
I20260812 19:07:09.743419 29572 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.744663 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet c89f8a2f65164cd08d1e760f9005e496 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:09.747239 30684 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.750800 31795 raft_consensus.cc:1050] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:09.753000 31795 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:09.766921 29572 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.773883 29459 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:09.768935 30681 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.775234 32053 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.776371 31795 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.778505 29335 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:09.781067 30681 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:09.782109 31933 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:09.782793 31877 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.788635 32056 tablet_replica.cc:330] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:09.789775 32056 raft_consensus.cc:2229] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:09.789531 31926 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:09.790616 32056 raft_consensus.cc:2258] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:09.791985 29572 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:09.797627 32056 ts_tablet_manager.cc:1905] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:09.797621 31795 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260812 19:07:09.804037 29450 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.818671 31935 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:09.848152 30685 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.855438 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:09.863176 31933 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:09.865847 32053 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.879194 32056 ts_tablet_manager.cc:1918] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:09.882236 32056 log.cc:1196] T c89f8a2f65164cd08d1e760f9005e496 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/c89f8a2f65164cd08d1e760f9005e496
I20260812 19:07:09.896232 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet c89f8a2f65164cd08d1e760f9005e496 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:09.913828 30681 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.927930 31779 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:09.911600 29174 catalog_manager.cc:5495] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.961400 29569 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:09.962229 29569 raft_consensus.cc:595] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:09.970753 29341 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:09.971329 29341 raft_consensus.cc:595] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:09.979539 31976 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.984238 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:09.984910 29450 raft_consensus.cc:595] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:09.990464 29341 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:09.995082 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:10.005671 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:10.016461 31779 raft_consensus.cc:979] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:10.006381 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:10.014474 29575 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.027107 29335 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:10.030176 29569 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:10.092857 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 143570us
I20260812 19:07:10.028033 31933 raft_consensus.cc:1067] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:10.113829 31926 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260812 19:07:10.115290 32060 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260812 19:07:10.132849 29335 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.165587 29577 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:10.167716 32093 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260812 19:07:10.193399 32102 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.193846 30681 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.199141 29576 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.200637 29174 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:10.224241 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e87477250ac44a318f7c0aef2d89fa36 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:10.237100 29578 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.245298 30681 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260812 19:07:10.250368 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 183087us
I20260812 19:07:10.278105 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:10.278679 29450 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:10.279060 29450 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:10.285199 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 223886us
I20260812 19:07:10.291662 29174 catalog_manager.cc:5495] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:10.315111 29450 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.315920 29445 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.334830 29450 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:10.345295 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:10.345909 30681 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.359046 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:10.359694 29337 raft_consensus.cc:3032] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:10.359944 29337 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:10.360661 29337 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.361456 30681 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:10.363830 32080 raft_consensus.cc:979] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:10.364506 32080 raft_consensus.cc:1067] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:10.361760 29337 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.377882 29452 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.289664 32093 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260812 19:07:10.397610 29337 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:10.407238 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:10.407814 29335 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:10.408164 29335 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.414645 29403 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:10.417874 30684 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:10.421697 29335 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.423517 29335 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:10.427541 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:10.428436 29567 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:10.428869 29567 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:10.430007 32088 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:10.430745 32088 raft_consensus.cc:686] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:10.431226 29342 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:10.431037 32081 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260812 19:07:10.431366 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:10.437355 29177 catalog_manager.cc:5495] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.442430 30681 raft_consensus.cc:2932] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.441262 31781 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260812 19:07:10.436159 30684 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.432147 32088 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.431905 29567 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.457048 31935 raft_consensus.cc:979] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:10.457659 32081 raft_consensus.cc:1067] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:10.459555 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:10.460038 29567 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.460049 29335 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:10.462998 29335 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.469213 31781 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.489737 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet e87477250ac44a318f7c0aef2d89fa36 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:10.490770 29341 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.494135 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 647dfc8bad934787a924a06988dda474 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:10.500643 30684 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:10.502475 29338 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.516614 30685 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:10.517519 30686 raft_consensus.cc:2932] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.525902 32115 tablet_replica.cc:330] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:10.534376 29335 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:10.536857 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:10.537449 29450 raft_consensus.cc:3032] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:10.537755 29450 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:10.527770 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:10.538560 29450 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:10.539479 32093 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:10.539788 29450 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.540284 31933 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:10.540594 29578 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:10.543144 32093 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:10.544076 32093 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:10.548281 29177 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.549942 29567 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:10.559692 31933 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:10.561789 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 647dfc8bad934787a924a06988dda474 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:10.579007 29450 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:10.580214 29282 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:10.581064 32102 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:10.580678 31933 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.581872 32102 raft_consensus.cc:686] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:10.582787 32102 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:10.593801 29174 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:10.597548 30686 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.598060 32116 tablet_replica.cc:330] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:10.601876 32116 raft_consensus.cc:2229] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:10.602720 32116 raft_consensus.cc:2258] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:10.604532 29335 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:10.605523 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet db322776a340462b91983d8e448cfe39 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:10.608356 32115 raft_consensus.cc:2229] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:10.609141 32115 raft_consensus.cc:2258] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:10.612951 32115 ts_tablet_manager.cc:1905] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:10.613283 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:10.614166 29567 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:10.616824 29578 raft_consensus.cc:2932] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.621356 29179 catalog_manager.cc:5495] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:10.616063 29177 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:10.652241 32115 ts_tablet_manager.cc:1918] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:10.652861 32115 log.cc:1196] T e87477250ac44a318f7c0aef2d89fa36 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/e87477250ac44a318f7c0aef2d89fa36
I20260812 19:07:10.653803 29567 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:10.657886 32116 ts_tablet_manager.cc:1905] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:10.655097 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet e87477250ac44a318f7c0aef2d89fa36 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:10.664736 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet db322776a340462b91983d8e448cfe39 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:10.666536 32115 tablet_replica.cc:330] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:10.668412 32115 raft_consensus.cc:2229] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:10.669504 32115 raft_consensus.cc:2258] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:10.678931 32115 ts_tablet_manager.cc:1905] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:10.685842 29174 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:10.695427 32115 ts_tablet_manager.cc:1918] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:10.695986 32115 log.cc:1196] T db322776a340462b91983d8e448cfe39 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/db322776a340462b91983d8e448cfe39
I20260812 19:07:10.698416 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet db322776a340462b91983d8e448cfe39 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:10.708379 32116 ts_tablet_manager.cc:1918] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:10.709081 32116 log.cc:1196] T 647dfc8bad934787a924a06988dda474 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/647dfc8bad934787a924a06988dda474
I20260812 19:07:10.711486 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 647dfc8bad934787a924a06988dda474 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:10.856570 29338 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:10.857906 32088 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:10.869417 30681 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:10.870900 32088 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:10.957496 30686 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:10.958536 32096 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:10.965911 29567 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:10.967550 32093 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:11.088266 29567 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:11.089506 32096 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:11.106391 29452 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:11.118886 32102 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:11.232030 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:11.267580 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:11.268052 29337 raft_consensus.cc:595] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:11.270141 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:11.270660 29337 raft_consensus.cc:595] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:11.272867 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:11.273284 29452 raft_consensus.cc:595] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:11.275350 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:11.275740 29452 raft_consensus.cc:595] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:11.276484 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260812 19:07:11.277562 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:11.278152 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:11.356297 32102 raft_consensus.cc:979] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:11.356807 31933 raft_consensus.cc:1067] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:11.357976 30681 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:11.358431 30681 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:11.358712 30681 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.364666 30681 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:11.366175 30681 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:11.366847 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:11.367443 29337 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:11.367736 29337 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:11.368005 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:11.368516 29578 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.368301 29337 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:11.369441 29337 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.374368 29578 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:11.374971 29337 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:11.375465 30623 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:11.376616 31954 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:11.377049 31954 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:11.377843 31954 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:11.393651 29174 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:11.448982 32088 raft_consensus.cc:979] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:11.449539 31976 raft_consensus.cc:1067] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:11.451195 29578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:11.451613 29578 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:11.451923 29578 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.457554 29578 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.459332 29578 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:11.459996 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:11.460165 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:11.460510 29338 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.461170 29450 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:11.461468 29450 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:11.462121 29450 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.462994 29450 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.465379 29338 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:11.466423 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:11.467152 32101 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:11.468259 29450 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:11.468428 32101 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:11.469388 32101 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.481428 29174 catalog_manager.cc:5495] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:11.500161 32102 raft_consensus.cc:979] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:11.500566 31933 raft_consensus.cc:1067] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:11.501832 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:11.502394 29450 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:11.502694 29450 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.508457 29450 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.510279 29450 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:11.511019 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:11.511577 29338 raft_consensus.cc:3032] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:11.511775 29338 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:11.511689 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:11.512308 29578 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.512420 29338 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.513403 29338 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:11.516973 29578 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:11.517740 29338 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:11.517843 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:11.518709 31976 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:11.519080 31976 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:11.519826 31976 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.528518 29174 catalog_manager.cc:5495] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:11.701874 29450 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:11.710158 29338 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:11.711339 31976 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:11.713382 29578 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:11.714280 29399 consensus_peers.cc:487] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: da12498df45141cd80e5cb36e68775c5. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:11.716082 31976 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:11.721714 32081 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.727245 29337 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.730224 29567 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.736084 29338 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:11.751857 29450 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:11.751375 29174 catalog_manager.cc:5495] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:07:11.755798 29281 consensus_peers.cc:487] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 69508446a398466a943e7ad290aa8e0c. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:11.757418 32093 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.759800 29567 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:11.763224 32093 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.768834 31933 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.771551 29450 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.805846 29179 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.812990 29452 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:11.812628 29567 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.819913 29578 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:11.821276 32152 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260812 19:07:11.822355 29399 consensus_peers.cc:487] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ea822fc3c0f456ab7c3f8d71f84ff27. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:11.827536 29338 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:11.829710 32081 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.833096 31976 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.834226 29578 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.850121 32081 raft_consensus.cc:979] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:11.850739 32081 raft_consensus.cc:1067] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:11.851161 29174 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.859037 29452 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:11.860337 29578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:11.860893 29578 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:11.861241 29578 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:11.867003 29338 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.872220 29578 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.874636 29578 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:11.875697 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:11.876344 29341 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:11.879528 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:11.880131 29450 raft_consensus.cc:3032] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:11.880398 29450 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:11.881084 29450 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.882217 29450 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:11.883635 29578 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260812 19:07:11.887573 29399 consensus_peers.cc:487] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fd63b9965cdb4b568fd3b2860f8c2e61. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:11.894119 29450 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:11.893558 29338 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:11.898170 31976 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.901790 29341 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:11.902927 29530 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:11.903479 29452 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:11.904367 31976 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.904470 32101 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:11.907251 32143 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.908591 32101 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:11.909417 32101 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:11.920142 29567 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:11.922257 31954 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:11.931422 29341 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:11.932950 31976 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.941612 31976 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.957796 29338 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:11.963930 29337 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:11.965557 31926 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:11.971340 29578 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:12.018651 29174 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:12.030112 29567 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:12.056273 29179 catalog_manager.cc:5495] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 2 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:12.061556 32081 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:12.076086 29338 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260812 19:07:12.085990 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 43693us
I20260812 19:07:12.085830 32178 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260812 19:07:12.098795 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 48142.4us
W20260812 19:07:12.111130 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 68882.6us
I20260812 19:07:12.214460 32223 ts_tablet_manager.cc:927] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:12.216410 32223 tablet_copy_client.cc:316] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:12.218271 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:12.218806 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-2ea822fc3c0f456ab7c3f8d71f84ff27
I20260812 19:07:12.232589 29474 tablet_copy_source_session.cc:205] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:12.237485 32223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:12.261503 32223 tablet_copy_client.cc:791] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:12.262126 32223 tablet_copy_client.cc:655] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:12.265659 32223 tablet_copy_client.cc:531] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:12.276970 32223 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:12.299994 32227 ts_tablet_manager.cc:927] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:12.307771 32227 tablet_copy_client.cc:316] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:12.311815 32223 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:12.311944 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet da12498df45141cd80e5cb36e68775c5 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:12.312549 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet da12498df45141cd80e5cb36e68775c5 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-da12498df45141cd80e5cb36e68775c5
I20260812 19:07:12.312784 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:12.320441 29474 tablet_copy_source_session.cc:205] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:12.312753 32223 tablet_bootstrap.cc:492] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:12.322767 32223 ts_tablet_manager.cc:1397] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.046s user 0.022s sys 0.005s
I20260812 19:07:12.323956 29445 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:12.324213 32227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da12498df45141cd80e5cb36e68775c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:12.325584 31926 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:12.325546 32223 raft_consensus.cc:348] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.335279 32223 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:12.336062 32223 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.342819 32223 ts_tablet_manager.cc:1428] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260812 19:07:12.346294 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-2ea822fc3c0f456ab7c3f8d71f84ff27 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:12.346728 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-2ea822fc3c0f456ab7c3f8d71f84ff27 on tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:12.354867 32223 ts_tablet_manager.cc:927] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:12.362998 29337 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:12.364228 32227 tablet_copy_client.cc:791] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:12.364923 32227 tablet_copy_client.cc:655] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:12.364774 32143 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:12.374758 32223 tablet_copy_client.cc:316] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:12.376530 32227 tablet_copy_client.cc:531] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:12.376479 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 69508446a398466a943e7ad290aa8e0c from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:12.379827 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 69508446a398466a943e7ad290aa8e0c from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-69508446a398466a943e7ad290aa8e0c
I20260812 19:07:12.389298 29355 tablet_copy_source_session.cc:205] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:12.393466 32223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69508446a398466a943e7ad290aa8e0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:12.397311 32227 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:12.427599 32223 tablet_copy_client.cc:791] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:12.429822 32223 tablet_copy_client.cc:655] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:12.433755 32223 tablet_copy_client.cc:531] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:12.435073 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:12.435705 29578 raft_consensus.cc:595] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:12.438741 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "647dfc8bad934787a924a06988dda474"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:12.439391 29450 raft_consensus.cc:595] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:12.443485 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:12.444015 29578 raft_consensus.cc:595] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:12.445219 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:12.445343 32223 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:12.447160 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:12.447767 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:12.448683 32227 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:12.449750 32227 tablet_bootstrap.cc:492] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:12.450479 32227 ts_tablet_manager.cc:1397] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.054s user 0.018s sys 0.009s
I20260812 19:07:12.453222 32227 raft_consensus.cc:348] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.454110 32227 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:12.454761 32227 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.467392 32227 ts_tablet_manager.cc:1428] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260812 19:07:12.477169 32231 ts_tablet_manager.cc:927] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:12.481568 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-da12498df45141cd80e5cb36e68775c5 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:12.488065 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-da12498df45141cd80e5cb36e68775c5 on tablet da12498df45141cd80e5cb36e68775c5 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:12.496685 32231 tablet_copy_client.cc:316] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:12.499629 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet fd63b9965cdb4b568fd3b2860f8c2e61 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:12.500157 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet fd63b9965cdb4b568fd3b2860f8c2e61 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-fd63b9965cdb4b568fd3b2860f8c2e61
I20260812 19:07:12.508526 29474 tablet_copy_source_session.cc:205] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:12.511718 32231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd63b9965cdb4b568fd3b2860f8c2e61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:12.520669 32223 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:12.528169 32231 tablet_copy_client.cc:791] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:12.528795 32231 tablet_copy_client.cc:655] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:12.538846 32223 tablet_bootstrap.cc:492] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:12.538879 32231 tablet_copy_client.cc:531] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:12.539759 32223 ts_tablet_manager.cc:1397] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.095s user 0.025s sys 0.024s
I20260812 19:07:12.540577 32217 raft_consensus.cc:979] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:12.541077 32088 raft_consensus.cc:1067] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:12.542732 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "647dfc8bad934787a924a06988dda474"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:12.543289 29338 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:12.543632 29338 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.545269 32223 raft_consensus.cc:348] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.547001 32223 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:12.547667 32223 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.551481 32231 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:12.551877 32223 ts_tablet_manager.cc:1428] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260812 19:07:12.554488 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-69508446a398466a943e7ad290aa8e0c received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:12.554911 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-69508446a398466a943e7ad290aa8e0c on tablet 69508446a398466a943e7ad290aa8e0c with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:12.563335 29338 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.572206 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:12.572796 29452 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:12.573041 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:12.573658 30681 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.573092 29452 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:12.586521 30681 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:12.586643 29452 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.587755 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:12.587834 29452 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.588649 32102 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:12.589103 32102 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:12.589919 32102 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.587846 29338 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:12.595391 29452 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:12.596046 32231 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:12.596946 32231 tablet_bootstrap.cc:492] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:12.597620 32231 ts_tablet_manager.cc:1397] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.046s user 0.028s sys 0.004s
I20260812 19:07:12.600535 32143 raft_consensus.cc:979] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:12.600979 32143 raft_consensus.cc:1067] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:12.601590 32231 raft_consensus.cc:348] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.602578 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:12.602887 32231 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:12.603494 29450 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:12.603916 29450 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:12.603658 32231 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:12.606945 32231 ts_tablet_manager.cc:1428] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260812 19:07:12.608853 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-fd63b9965cdb4b568fd3b2860f8c2e61 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:12.609262 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-fd63b9965cdb4b568fd3b2860f8c2e61 on tablet fd63b9965cdb4b568fd3b2860f8c2e61 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:12.617157 29450 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.620472 29450 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:12.622509 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:12.625622 29578 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:12.625931 29578 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:12.626667 29578 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.627825 29578 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:12.628152 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:12.628726 30686 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:12.634776 29578 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:12.635998 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:12.636919 32088 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:12.637416 32088 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:12.638103 29174 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:12.638311 32088 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:12.645897 30686 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:12.660365 29177 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:12.691059 30681 raft_consensus.cc:1203] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 19:07:12.698642 30681 raft_consensus.cc:1203] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:12.746341 30681 raft_consensus.cc:1203] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 19:07:12.766847 32207 raft_consensus.cc:979] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:12.767416 32101 raft_consensus.cc:1067] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:12.768749 29452 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:12.769255 29452 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:12.769582 29452 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.783001 29452 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:12.785642 29452 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:12.785821 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:12.786504 29338 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.785823 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:12.790582 29576 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:12.790910 29576 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:12.791628 29576 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:12.792894 29576 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:12.801080 29338 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:12.804064 29576 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:12.805271 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:12.806128 32088 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:12.806602 32088 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:12.807415 32088 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:12.824242 29176 catalog_manager.cc:5495] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:12.872520 30681 raft_consensus.cc:1203] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 19:07:13.015848 29445 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.016944 32203 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.040928 30681 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.042359 32203 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.145515 32081 raft_consensus.cc:1050] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:13.148337 32081 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.157091 29578 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:13.158357 29338 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:13.158284 32081 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:13.159636 30681 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:13.160651 30684 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:13.160948 32217 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:13.161834 32081 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:13.162197 32088 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:13.170455 32217 raft_consensus.cc:1050] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:13.173249 32217 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.206643 29578 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.228523 29578 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:13.229646 29176 catalog_manager.cc:5495] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.211083 32271 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.235044 32088 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:13.235810 29576 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:13.244721 32274 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:13.251693 29338 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.254118 29567 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.256052 32272 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.258584 30681 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.260181 32081 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.264508 29335 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.274113 32081 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.280737 30684 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.334720 32204 raft_consensus.cc:1050] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:13.337277 32204 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.377408 32271 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.383306 30681 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.387132 29337 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:13.390236 29445 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.406208 32274 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:13.411605 30684 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:13.412922 32274 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:13.416180 29578 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:13.421187 32272 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.424794 30684 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.426077 32274 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.428443 29578 raft_consensus.cc:2932] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.432132 29451 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.435854 29576 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.440727 32217 raft_consensus.cc:1050] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:13.442633 32290 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.443452 32217 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.446343 29337 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:13.469102 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 63754.4us
I20260812 19:07:13.468423 29176 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.463853 29177 catalog_manager.cc:5495] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.474726 29576 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.480254 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet da12498df45141cd80e5cb36e68775c5 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:13.481436 30684 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.483312 32311 tablet_replica.cc:330] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:13.484364 32311 raft_consensus.cc:2229] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:13.485072 32311 raft_consensus.cc:2258] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:13.491533 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet da12498df45141cd80e5cb36e68775c5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:13.491667 32311 ts_tablet_manager.cc:1905] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:13.495263 29450 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.500396 29337 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.501715 30684 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.504237 29579 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:13.508188 30684 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.509119 32279 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:13.511574 32088 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.512473 32274 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.514343 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:13.517017 32276 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:13.518172 29337 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.519830 32276 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.531682 32088 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.541182 30684 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.546254 29338 raft_consensus.cc:2932] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.548586 32311 ts_tablet_manager.cc:1918] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:13.553450 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:13.563284 32299 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:13.576191 32204 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:13.576922 32274 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.579027 32311 log.cc:1196] T da12498df45141cd80e5cb36e68775c5 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/da12498df45141cd80e5cb36e68775c5
I20260812 19:07:13.588212 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:13.588858 29450 raft_consensus.cc:595] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:13.589443 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet da12498df45141cd80e5cb36e68775c5 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:13.599227 32299 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.587755 29337 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.595263 29174 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 4 to 5, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.611742 29445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:13.602478 30681 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.612370 29445 raft_consensus.cc:595] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:13.615541 29578 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.616750 30686 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.623207 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:13.623377 29338 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:13.623894 29338 raft_consensus.cc:595] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:13.642210 29445 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.646186 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:13.653211 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:13.653901 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:13.659492 29174 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.683521 32317 tablet_replica.cc:330] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
W20260812 19:07:13.687812 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 61294.5us
I20260812 19:07:13.693897 29177 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.696117 32317 raft_consensus.cc:2229] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:13.698556 29578 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.698518 32317 raft_consensus.cc:2258] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:13.713811 32317 ts_tablet_manager.cc:1905] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:13.721460 29337 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.734530 29579 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.734694 30686 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.736199 32299 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.737118 32204 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.748353 30686 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.744230 32204 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.759797 29578 raft_consensus.cc:2932] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.768147 29452 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.768792 32317 ts_tablet_manager.cc:1918] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260812 19:07:13.769198 32317 log.cc:1196] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/2ea822fc3c0f456ab7c3f8d71f84ff27
I20260812 19:07:13.770614 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 69508446a398466a943e7ad290aa8e0c with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:13.771484 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 2ea822fc3c0f456ab7c3f8d71f84ff27 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:13.777443 29177 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.780145 30686 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.782541 32088 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.782963 29579 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:13.791060 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 69508446a398466a943e7ad290aa8e0c with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:13.800209 32323 tablet_replica.cc:330] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:13.800562 32217 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.802556 32274 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:13.804348 30686 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.808724 32323 raft_consensus.cc:2229] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:13.809546 32323 raft_consensus.cc:2258] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:13.812850 32323 ts_tablet_manager.cc:1905] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:13.822448 29578 raft_consensus.cc:2932] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.824681 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet fd63b9965cdb4b568fd3b2860f8c2e61 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:13.826762 29174 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 4 to 5, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:13.838825 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet fd63b9965cdb4b568fd3b2860f8c2e61 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:13.839764 32311 tablet_replica.cc:330] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:13.840742 32311 raft_consensus.cc:2229] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:13.841516 32311 raft_consensus.cc:2258] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:13.845638 32311 ts_tablet_manager.cc:1905] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:13.854559 32323 ts_tablet_manager.cc:1918] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260812 19:07:13.855043 32323 log.cc:1196] T 69508446a398466a943e7ad290aa8e0c P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/69508446a398466a943e7ad290aa8e0c
I20260812 19:07:13.856810 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 69508446a398466a943e7ad290aa8e0c (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:13.861990 32311 ts_tablet_manager.cc:1918] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260812 19:07:13.862429 32311 log.cc:1196] T fd63b9965cdb4b568fd3b2860f8c2e61 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/fd63b9965cdb4b568fd3b2860f8c2e61
I20260812 19:07:13.863996 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet fd63b9965cdb4b568fd3b2860f8c2e61 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:13.887511 32276 raft_consensus.cc:979] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:13.888077 32274 raft_consensus.cc:1067] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:13.894215 29567 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:13.894722 29567 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:13.894958 29567 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:13.899529 32274 raft_consensus.cc:979] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:13.899948 32217 raft_consensus.cc:1067] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:13.900079 29567 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:13.901254 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34288
I20260812 19:07:13.901774 30686 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:13.902081 29567 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:13.902444 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:13.902132 30686 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:13.902997 29338 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:13.909272 30686 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.910356 29338 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:13.911697 29530 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:13.911846 30686 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:13.912956 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:13.913553 29452 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:13.913838 29452 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:13.914491 29452 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:13.915220 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:13.915577 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:13.916447 29567 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:13.912963 32319 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:13.915772 29445 raft_consensus.cc:3032] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:13.917752 29445 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:13.918637 29445 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.919889 29445 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:13.915702 29452 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:13.917809 32319 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:13.925200 32319 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:13.930116 29452 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:13.935032 29445 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:13.936084 30625 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:13.936843 32202 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:13.937340 32202 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:13.938175 32202 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.942669 29567 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:13.939291 29177 catalog_manager.cc:5495] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.964643 29177 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 3 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:13.969400 32299 raft_consensus.cc:979] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:13.969995 32290 raft_consensus.cc:1067] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:13.971688 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:13.972251 29450 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:13.972589 29450 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:13.978914 29450 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.981459 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:13.982102 29338 raft_consensus.cc:3032] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:13.982414 29338 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:13.983117 29338 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:13.984244 29338 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:13.984876 29450 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:13.985857 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:13.986505 30681 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:13.990788 29338 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:13.991981 29403 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:13.992661 32274 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:13.993129 32274 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:13.993000 30681 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:13.994164 32274 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:14.003592 29177 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:14.367530 29450 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:14.368800 32328 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:14.382956 29567 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:14.384609 32328 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:14.402287 30681 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:14.403492 32276 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:14.416584 29338 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:14.417832 32274 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:14.426591 29450 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:14.431463 32319 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:14.457051 29338 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:14.458510 32319 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:14.850041 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:14.898667 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:14.899196 29578 raft_consensus.cc:595] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:14.900691 29338 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:14.901724 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:14.902300 29450 raft_consensus.cc:595] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:14.903420 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260812 19:07:14.905141 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:14.905797 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:14.915813 29578 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 19:07:14.917310 29452 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260812 19:07:14.915922 29281 consensus_peers.cc:487] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7f7d48e270c456b9b7eeab6f3165a89. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:14.918633 32299 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:14.919548 32290 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:14.930476 32299 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:14.935668 29578 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:14.944468 29452 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:14.964562 29176 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 3, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:14.971179 29450 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:14.986881 29578 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:14.988226 32217 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:14.989030 29338 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260812 19:07:14.990562 32217 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260812 19:07:14.991523 29399 consensus_peers.cc:487] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 514939f267f2450faf798117d94726ab. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:15.060365 32357 raft_consensus.cc:979] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:15.061201 32342 raft_consensus.cc:1067] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:15.061268 32369 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.063788 29578 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.067533 30686 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34302
I20260812 19:07:15.068123 30686 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:15.068473 30686 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:15.080613 30686 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:15.083166 29337 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.092069 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:15.092748 29338 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:15.094314 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:15.094901 29567 raft_consensus.cc:3032] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:15.095232 29567 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:15.095888 29567 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:15.096915 29567 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:15.101727 29338 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:15.101513 29177 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.102823 30629 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:15.105324 32180 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:15.105769 32180 raft_consensus.cc:686] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:15.106638 32180 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:15.101670 30686 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:15.120129 29578 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.104148 29567 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:15.124243 29174 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:15.139148 29452 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:15.140336 29527 consensus_peers.cc:487] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 47762c70d86d4362baf4ba75331efe8a. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:15.142002 32362 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:15.146951 29338 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:15.149307 32362 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:15.151676 32369 raft_consensus.cc:979] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:15.152215 32365 raft_consensus.cc:1067] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:15.151358 32357 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.153571 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:15.154109 29338 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:15.154407 29338 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:15.154141 29450 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.161556 29338 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:15.162853 29337 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.172063 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:15.172178 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:15.172736 29576 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:15.173050 29445 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:15.173370 29445 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:15.174098 29445 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:15.174540 29257 auto_rebalancer.cc:254] failed to send replica move request: Remote error: Service unavailable: leader transfer in progress
I20260812 19:07:15.175239 29445 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:15.176833 29338 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:15.180161 29176 catalog_manager.cc:5495] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:15.188753 29445 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:15.190057 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:15.190881 32299 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:15.191373 32299 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:15.192171 32299 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:15.198324 29576 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:15.199447 29257 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet f55b37aa5102423ebbccef3f872d85e8, TS 6dd313c6592d4f33bab314a73f28b007 -> TS a34c47dd29c14748a018d716ac6c39df move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:15.199826 29257 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet f55b37aa5102423ebbccef3f872d85e8, TS 6dd313c6592d4f33bab314a73f28b007 -> TS a34c47dd29c14748a018d716ac6c39df move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:15.213460 29176 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:15.493603 32380 ts_tablet_manager.cc:927] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:15.497394 32380 tablet_copy_client.cc:316] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:15.504838 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet c7f7d48e270c456b9b7eeab6f3165a89 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:15.505543 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet c7f7d48e270c456b9b7eeab6f3165a89 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-c7f7d48e270c456b9b7eeab6f3165a89
I20260812 19:07:15.527009 29578 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:15.527666 29355 tablet_copy_source_session.cc:205] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:15.528493 32180 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:15.531636 32380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7f7d48e270c456b9b7eeab6f3165a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:15.558732 29338 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:15.573251 32180 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260812 19:07:15.599345 32380 tablet_copy_client.cc:791] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:15.600114 32380 tablet_copy_client.cc:655] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:15.612468 32380 tablet_copy_client.cc:531] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:15.622879 32380 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:15.644429 32390 ts_tablet_manager.cc:927] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:15.654667 32390 tablet_copy_client.cc:316] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:15.657920 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 514939f267f2450faf798117d94726ab from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:15.658560 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 514939f267f2450faf798117d94726ab from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-514939f267f2450faf798117d94726ab
I20260812 19:07:15.667318 29474 tablet_copy_source_session.cc:205] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:15.669086 32380 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:15.670281 32380 tablet_bootstrap.cc:492] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:15.670259 32390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 514939f267f2450faf798117d94726ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:15.673290 32380 ts_tablet_manager.cc:1397] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.051s user 0.014s sys 0.018s
I20260812 19:07:15.676968 32392 ts_tablet_manager.cc:927] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:15.677794 32380 raft_consensus.cc:348] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.680720 32380 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:15.681438 32380 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.691906 32380 ts_tablet_manager.cc:1428] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.018s user 0.008s sys 0.001s
I20260812 19:07:15.695127 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-c7f7d48e270c456b9b7eeab6f3165a89 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:15.695530 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-c7f7d48e270c456b9b7eeab6f3165a89 on tablet c7f7d48e270c456b9b7eeab6f3165a89 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:15.698412 32390 tablet_copy_client.cc:791] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:15.699026 32390 tablet_copy_client.cc:655] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:15.707294 29567 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260812 19:07:15.714635 32390 tablet_copy_client.cc:531] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:15.717460 32386 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:15.726732 32390 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:15.743656 32392 tablet_copy_client.cc:316] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:15.748171 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 47762c70d86d4362baf4ba75331efe8a from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:15.748795 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 47762c70d86d4362baf4ba75331efe8a from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-47762c70d86d4362baf4ba75331efe8a
I20260812 19:07:15.754607 32390 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:15.755584 29445 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260812 19:07:15.755585 32390 tablet_bootstrap.cc:492] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:15.757073 32390 ts_tablet_manager.cc:1397] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.013s
I20260812 19:07:15.760246 32390 raft_consensus.cc:348] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.761111 32390 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:15.761719 32390 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.764386 32390 ts_tablet_manager.cc:1428] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260812 19:07:15.765488 29601 tablet_copy_source_session.cc:205] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:15.768299 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-514939f267f2450faf798117d94726ab received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:15.768702 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-514939f267f2450faf798117d94726ab on tablet 514939f267f2450faf798117d94726ab with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:15.769719 32392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47762c70d86d4362baf4ba75331efe8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:15.778411 32349 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:15.804695 32392 tablet_copy_client.cc:791] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:15.805368 32392 tablet_copy_client.cc:655] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:15.814217 32392 tablet_copy_client.cc:531] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:15.822796 32392 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:15.853039 32392 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:15.853741 32392 tablet_bootstrap.cc:492] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:15.854417 32392 ts_tablet_manager.cc:1397] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.032s user 0.014s sys 0.013s
I20260812 19:07:15.856895 32392 raft_consensus.cc:348] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.857697 32392 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:15.858247 32392 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:15.860345 32392 ts_tablet_manager.cc:1428] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.006s user 0.001s sys 0.003s
I20260812 19:07:15.871030 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-47762c70d86d4362baf4ba75331efe8a received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:15.871460 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-47762c70d86d4362baf4ba75331efe8a on tablet 47762c70d86d4362baf4ba75331efe8a with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:15.956892 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:16.005663 29342 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:16.006347 29342 raft_consensus.cc:595] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:16.007570 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:16.009104 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:16.009624 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:16.065651 30686 raft_consensus.cc:1203] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:16.076221 30686 raft_consensus.cc:1203] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260812 19:07:16.215993 30686 raft_consensus.cc:1203] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20260812 19:07:16.332407 32386 raft_consensus.cc:979] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:16.332994 32299 raft_consensus.cc:1067] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:16.334909 29579 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57270
I20260812 19:07:16.335906 29579 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:16.336268 29579 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:16.342093 29579 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.344386 29579 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:16.345119 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:16.345049 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:16.345822 29335 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:16.346197 29335 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:16.346637 30686 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:16.346941 29335 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.348150 29335 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:16.353936 29335 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:16.354193 30686 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:16.355536 29527 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:16.356503 32368 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:16.357003 32368 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:16.357931 32368 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.370345 29174 catalog_manager.cc:5495] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:16.486119 32368 raft_consensus.cc:1050] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:16.488312 32368 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.493852 30686 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:16.495791 32357 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:16.496907 29335 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:16.498621 29452 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:16.499336 32362 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:16.502202 32362 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:16.506146 32299 raft_consensus.cc:1050] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:16.516341 32299 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.529301 29567 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:16.531810 32409 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:16.539505 29450 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:16.546248 32386 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:16.547935 30686 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 19:07:16.549463 32409 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:16.553864 32357 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.561594 29450 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.566447 29338 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.579550 30681 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.589393 29174 catalog_manager.cc:5495] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.620675 29576 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.634193 30681 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:16.635500 32357 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:16.636103 30681 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.637072 32299 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.638654 29452 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.642867 32362 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.654240 30681 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.656391 29450 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:16.657193 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 47762c70d86d4362baf4ba75331efe8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:16.653410 29177 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.664279 32342 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:16.670208 29176 catalog_manager.cc:5495] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:16.671339 29578 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.684388 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 47762c70d86d4362baf4ba75331efe8a with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:16.695783 29452 raft_consensus.cc:2932] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.699664 29335 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.719308 32430 tablet_replica.cc:330] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:16.720927 32430 raft_consensus.cc:2229] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:16.722050 30681 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:16.723239 29578 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:16.724924 32299 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:16.725699 32409 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:16.723291 32430 raft_consensus.cc:2258] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:16.734939 32299 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.735642 29578 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.743463 32430 ts_tablet_manager.cc:1905] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:16.744139 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c7f7d48e270c456b9b7eeab6f3165a89 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:16.743992 30681 raft_consensus.cc:2932] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.765304 29177 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 4 to 5, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.781706 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet c7f7d48e270c456b9b7eeab6f3165a89 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:16.799157 32430 ts_tablet_manager.cc:1918] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:16.799693 32430 log.cc:1196] T 47762c70d86d4362baf4ba75331efe8a P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/47762c70d86d4362baf4ba75331efe8a
I20260812 19:07:16.801517 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 47762c70d86d4362baf4ba75331efe8a (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:16.802268 32431 tablet_replica.cc:330] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:16.803248 32431 raft_consensus.cc:2229] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:16.803969 32431 raft_consensus.cc:2258] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:16.807909 32431 ts_tablet_manager.cc:1905] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:16.821427 32431 ts_tablet_manager.cc:1918] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260812 19:07:16.821888 32431 log.cc:1196] T c7f7d48e270c456b9b7eeab6f3165a89 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/c7f7d48e270c456b9b7eeab6f3165a89
I20260812 19:07:16.824056 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet c7f7d48e270c456b9b7eeab6f3165a89 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:16.830013 32365 raft_consensus.cc:1050] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:16.832307 32365 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.840346 29576 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:16.841771 32436 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:16.842306 30681 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:16.842200 29338 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:16.844642 32365 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:16.845613 32436 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:16.866549 32365 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.871742 30686 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.879446 29578 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.880580 29335 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.886993 29177 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260812 19:07:16.904903 32434 consensus_queue.cc:1159] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dd313c6592d4f33bab314a73f28b007" responder_term: 3 status { last_received { term: 3 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 3 index: 5 } }
I20260812 19:07:16.906455 29445 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:16.918607 29335 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:16.921838 30686 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:16.923255 32434 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:16.923704 32357 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:16.924154 29335 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:16.925621 32445 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:16.931828 30681 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:16.933709 32362 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:16.954876 32443 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.958230 29337 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.959997 30681 raft_consensus.cc:2932] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:16.980816 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 514939f267f2450faf798117d94726ab with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:16.994809 29179 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:17.009153 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet 514939f267f2450faf798117d94726ab with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:17.011520 32456 tablet_replica.cc:330] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:17.012638 32456 raft_consensus.cc:2229] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:17.013464 32456 raft_consensus.cc:2258] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:17.016569 32456 ts_tablet_manager.cc:1905] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:17.058763 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:17.057994 32456 ts_tablet_manager.cc:1918] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260812 19:07:17.065841 32456 log.cc:1196] T 514939f267f2450faf798117d94726ab P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/514939f267f2450faf798117d94726ab
I20260812 19:07:17.067803 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet 514939f267f2450faf798117d94726ab (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:17.105764 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:17.106280 29578 raft_consensus.cc:595] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:17.107336 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:17.108812 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:17.109275 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:17.227663 32342 raft_consensus.cc:979] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:17.228127 32357 raft_consensus.cc:1067] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:17.229461 29445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:17.229990 29445 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:17.230250 29445 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:17.235688 29445 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:17.237015 29445 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:17.237854 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:17.237738 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:17.238433 30681 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:17.238730 29578 raft_consensus.cc:3032] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:17.239040 29578 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:17.239749 29578 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:17.240775 29578 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:17.243892 30681 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:17.245010 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:17.245805 29578 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:17.247676 32434 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:17.248150 32434 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:17.248960 32434 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:17.265929 29176 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:17.646281 29576 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:17.647492 32445 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:17.657151 30686 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:17.658716 32434 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:18.145846 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:18.197968 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:18.198607 29450 raft_consensus.cc:595] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:18.200132 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:18.201901 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:18.202543 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:18.214994 29338 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:07:18.228845 29281 consensus_peers.cc:487] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f55b37aa5102423ebbccef3f872d85e8. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:18.231580 29576 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:07:18.233220 32349 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:18.244737 29445 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:07:18.246332 32409 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:18.277809 32409 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.286837 29445 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.313038 29578 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.337477 32445 raft_consensus.cc:979] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:18.337244 29576 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:18.337971 32445 raft_consensus.cc:1067] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:18.342595 29179 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 4, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:18.358448 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "514939f267f2450faf798117d94726ab"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:18.359321 29337 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:18.359766 29337 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:18.370604 29337 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:18.385185 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:18.387286 29445 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
W20260812 19:07:18.389030 29527 consensus_peers.cc:487] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f6dae5f1cee3498787a4a28802d59d16. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:18.390057 29337 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:18.390247 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:18.390177 29338 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:18.390811 30686 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:18.393110 32362 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:18.388545 29445 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:18.397203 29445 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:18.400650 29445 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:18.403296 30686 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:18.409765 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:18.410778 32409 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:18.411262 32409 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:18.412179 32409 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:18.421460 29445 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:18.419118 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 53591.2us
I20260812 19:07:18.427129 29451 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:18.432403 32357 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:18.451570 29337 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.451138 32357 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:07:18.494480 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 58562.4us
I20260812 19:07:18.503058 29576 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:18.517853 29452 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:18.518838 29337 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:18.519414 32502 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:18.520473 32362 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260812 19:07:18.536893 29527 consensus_peers.cc:487] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8d9f6c46c1a5494aa914b41b3a1616a1. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:18.539476 29177 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:18.541868 29177 catalog_manager.cc:5495] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.551470 32502 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.552557 29337 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.584225 29177 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.577546 29451 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.593426 29452 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.617991 29451 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:18.644399 29576 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 19:07:18.645543 29337 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:18.647091 29399 consensus_peers.cc:487] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cbee78e8845445eb3a4cbba1399e988. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:18.647217 32481 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:18.648900 32481 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:18.667129 32481 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.682536 29579 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.714051 29337 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.742828 29176 catalog_manager.cc:5495] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:18.871384 32521 ts_tablet_manager.cc:927] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:18.873695 32522 ts_tablet_manager.cc:927] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:18.892462 30686 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:18.894179 29451 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:18.894990 32468 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:18.896117 32349 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:18.897109 32521 tablet_copy_client.cc:316] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:18.906194 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet f6dae5f1cee3498787a4a28802d59d16 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:18.907774 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet f6dae5f1cee3498787a4a28802d59d16 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-f6dae5f1cee3498787a4a28802d59d16
I20260812 19:07:18.978284 29601 tablet_copy_source_session.cc:205] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:18.980537 32522 tablet_copy_client.cc:316] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
W20260812 19:07:19.061045 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 96661.3us
W20260812 19:07:19.145597 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 68057.3us
I20260812 19:07:19.148895 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet f55b37aa5102423ebbccef3f872d85e8 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:19.149508 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet f55b37aa5102423ebbccef3f872d85e8 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-f55b37aa5102423ebbccef3f872d85e8
I20260812 19:07:19.215497 32521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6dae5f1cee3498787a4a28802d59d16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:19.212215 29355 tablet_copy_source_session.cc:205] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:19.249377 32522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f55b37aa5102423ebbccef3f872d85e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:19.393256 32521 tablet_copy_client.cc:791] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
W20260812 19:07:19.398571 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 91552.3us
W20260812 19:07:19.422057 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 66818.5us
I20260812 19:07:19.412253 32521 tablet_copy_client.cc:655] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:19.447062 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:19.462329 32558 ts_tablet_manager.cc:927] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:19.580981 32521 tablet_copy_client.cc:531] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:19.622773 32567 ts_tablet_manager.cc:927] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:19.635950 29338 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:19.636775 29338 raft_consensus.cc:595] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:19.642642 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:19.644559 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:19.645174 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:19.641639 32522 tablet_copy_client.cc:791] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
W20260812 19:07:19.679167 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 257015us
I20260812 19:07:19.689602 32521 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:19.693557 32522 tablet_copy_client.cc:655] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:19.737614 32522 tablet_copy_client.cc:531] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:19.745304 32558 tablet_copy_client.cc:316] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.67:44573
I20260812 19:07:19.762805 32567 tablet_copy_client.cc:316] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.66:43659
I20260812 19:07:19.790160 29601 tablet_copy_service.cc:140] P dd7e8c7a8bc54e73a39411cfc241fe9c: Received BeginTabletCopySession request for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:46208)
I20260812 19:07:19.798610 29601 tablet_copy_service.cc:161] P dd7e8c7a8bc54e73a39411cfc241fe9c: Beginning new tablet copy session on tablet 8d9f6c46c1a5494aa914b41b3a1616a1 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:46208: session id = a34c47dd29c14748a018d716ac6c39df-8d9f6c46c1a5494aa914b41b3a1616a1
I20260812 19:07:19.791911 32520 raft_consensus.cc:979] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:19.802471 32520 raft_consensus.cc:1067] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
W20260812 19:07:19.851015 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 44014.5us
I20260812 19:07:19.856077 29577 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57270
I20260812 19:07:19.856781 29577 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:19.857203 29577 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:19.855574 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49213.8us
I20260812 19:07:19.870213 29474 tablet_copy_service.cc:140] P 182784ca1eb74135b8f16d8894fa4b01: Received BeginTabletCopySession request for tablet 8cbee78e8845445eb3a4cbba1399e988 from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41204)
I20260812 19:07:19.870801 29474 tablet_copy_service.cc:161] P 182784ca1eb74135b8f16d8894fa4b01: Beginning new tablet copy session on tablet 8cbee78e8845445eb3a4cbba1399e988 from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41204: session id = a34c47dd29c14748a018d716ac6c39df-8cbee78e8845445eb3a4cbba1399e988
I20260812 19:07:19.888340 29474 tablet_copy_source_session.cc:205] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:19.904237 29601 tablet_copy_source_session.cc:205] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:19.906342 32522 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:19.889214 29577 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:19.911474 29577 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:19.912279 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:19.912848 29332 raft_consensus.cc:3032] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:19.913120 29332 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:19.913767 29332 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:19.914942 29332 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:19.916195 32558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d9f6c46c1a5494aa914b41b3a1616a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:19.926465 32567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cbee78e8845445eb3a4cbba1399e988. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:19.926517 29332 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:19.927848 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:19.928735 32541 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:19.929188 32541 raft_consensus.cc:686] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:19.930684 32541 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:19.933460 32521 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:19.936823 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:19.938127 30675 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:19.944882 32521 tablet_bootstrap.cc:492] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:19.990744 32521 ts_tablet_manager.cc:1397] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.301s user 0.023s sys 0.017s
I20260812 19:07:20.040656 30675 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:20.052579 29177 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 2 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:20.119154 32521 raft_consensus.cc:348] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:20.120942 32521 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:20.138501 32567 tablet_copy_client.cc:791] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:20.221788 32567 tablet_copy_client.cc:655] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:20.129051 32521 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:20.232050 32521 ts_tablet_manager.cc:1428] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.239s user 0.010s sys 0.014s
I20260812 19:07:20.263900 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-f6dae5f1cee3498787a4a28802d59d16 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:20.264649 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-f6dae5f1cee3498787a4a28802d59d16 on tablet f6dae5f1cee3498787a4a28802d59d16 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:20.277704 32567 tablet_copy_client.cc:531] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:20.342343 32558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:34302 (request call id 513) took 1069 ms. Trace:
I20260812 19:07:20.343189 32558 rpcz_store.cc:276] 0812 19:07:19.272447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:19.272607 (+ 160us) service_pool.cc:227] Handling call
0812 19:07:19.462972 (+190365us) ts_tablet_manager.cc:928] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
0812 19:07:19.915062 (+452090us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:07:20.082389 (+167327us) tablet_metadata.cc:641] Metadata flushed
0812 19:07:20.102159 (+ 19770us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:07:20.338325 (+236166us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:07:20.341427 (+ 3102us) tablet_replica.cc:205] Creating consensus instance
0812 19:07:20.342087 (+ 660us) ts_tablet_manager.cc:983] Replica registered.
0812 19:07:20.342195 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":52529,"tablet-copy.queue_time_us":148752,"thread_start_us":305470,"threads_started":2}
I20260812 19:07:20.354498 32558 tablet_copy_client.cc:791] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:20.355648 32558 tablet_copy_client.cc:655] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:20.368572 29342 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:20.378757 30686 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:20.380458 32542 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:20.412375 32558 tablet_copy_client.cc:531] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:20.400707 32541 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260812 19:07:20.458379 32567 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:20.492394 30675 raft_consensus.cc:1203] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:20.569773 32558 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
W20260812 19:07:20.673839 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 106570us
I20260812 19:07:20.692739 32522 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260812 19:07:20.705068 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 59389.7us
I20260812 19:07:20.749887 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:20.744038 32522 tablet_bootstrap.cc:492] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:20.789440 32522 ts_tablet_manager.cc:1397] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 0.883s user 0.035s sys 0.064s
W20260812 19:07:20.889993 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 63742.3us
I20260812 19:07:20.887411 32522 raft_consensus.cc:348] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:20.949118 32522 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:20.969604 32522 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:07:20.978610 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 47846.5us
I20260812 19:07:20.983021 32522 ts_tablet_manager.cc:1428] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.177s user 0.010s sys 0.002s
I20260812 19:07:20.993896 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-f55b37aa5102423ebbccef3f872d85e8 received from {username='slave'} at 127.0.0.1:41568
I20260812 19:07:20.994341 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-f55b37aa5102423ebbccef3f872d85e8 on tablet f55b37aa5102423ebbccef3f872d85e8 with peer a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:20.998601 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 71431us
W20260812 19:07:21.016887 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 104327us
W20260812 19:07:21.080749 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 56465.2us
I20260812 19:07:21.083527 30674 raft_consensus.cc:1203] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4] Dedup: 4.4->[]
W20260812 19:07:21.113839 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 183158us
W20260812 19:07:21.097146 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 71774us
I20260812 19:07:21.389226 29567 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:21.390043 29567 raft_consensus.cc:595] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
W20260812 19:07:21.403906 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 272078us
I20260812 19:07:21.421736 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:21.470809 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:21.471626 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260812 19:07:21.482759 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 62906us
I20260812 19:07:21.415418 32567 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:21.428082 32542 raft_consensus.cc:1050] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
W20260812 19:07:21.528348 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 85943.4us
I20260812 19:07:21.570881 32567 tablet_bootstrap.cc:492] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:21.572293 32567 ts_tablet_manager.cc:1397] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 1.114s user 0.029s sys 0.023s
W20260812 19:07:21.590905 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 80622.2us
W20260812 19:07:21.592557 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 90883.1us
I20260812 19:07:21.573112 32542 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:21.626127 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 111827us
W20260812 19:07:21.635998 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 136583us
W20260812 19:07:21.686537 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 57501.5us
W20260812 19:07:21.741677 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 109824us
W20260812 19:07:21.749380 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 61702.2us
W20260812 19:07:21.806308 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 299011us
I20260812 19:07:21.801858 32558 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 19:07:21.761020 32567 raft_consensus.cc:348] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:21.827003 32567 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
W20260812 19:07:21.889389 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47917us
I20260812 19:07:21.917486 32558 tablet_bootstrap.cc:492] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
W20260812 19:07:21.995419 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 64211.5us
W20260812 19:07:22.006104 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 86200.3us
I20260812 19:07:22.024452 29332 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:22.036763 29459 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260812 19:07:22.017462 32558 ts_tablet_manager.cc:1397] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 1.448s user 0.039s sys 0.032s
W20260812 19:07:22.048297 32637 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.250s user 0.002s sys 0.000s
I20260812 19:07:21.929692 32567 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:22.225689 32567 ts_tablet_manager.cc:1428] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.653s user 0.014s sys 0.008s
I20260812 19:07:22.307662 32605 raft_consensus.cc:979] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
W20260812 19:07:22.289403 32567 ts_tablet_manager.cc:1460] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Tablet startup took 714ms
I20260812 19:07:22.330389 32605 raft_consensus.cc:1067] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:22.336452 32349 raft_consensus.cc:1050] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
W20260812 19:07:22.326648 32567 ts_tablet_manager.cc:1462] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Trace:
0812 19:07:19.352157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:19.371642 (+ 19485us) service_pool.cc:227] Handling call
0812 19:07:19.635846 (+264204us) ts_tablet_manager.cc:928] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
0812 19:07:19.925280 (+289434us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:07:19.969580 (+ 44300us) tablet_metadata.cc:641] Metadata flushed
0812 19:07:19.970054 (+ 474us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:07:20.131462 (+161408us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:07:20.132043 (+ 581us) tablet_replica.cc:205] Creating consensus instance
0812 19:07:20.132601 (+ 558us) ts_tablet_manager.cc:983] Replica registered.
0812 19:07:20.132727 (+ 126us) inbound_call.cc:173] Queueing success response
0812 19:07:20.457763 (+325036us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0812 19:07:21.572862 (+1115099us) ts_tablet_manager.cc:1429] Starting tablet replica
0812 19:07:21.584345 (+ 11483us) tablet_replica.cc:259] Starting instrumentation
0812 19:07:21.585064 (+ 719us) tablet_replica.cc:278] Starting consensus
Metrics: {"dns_us":21039,"raft.queue_time_us":5749,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":2533,"tablet-copy.queue_time_us":248884,"tablet-download-pool.queue_time_us":218,"tablet-download-pool.run_cpu_time_us":1570,"tablet-download-pool.run_wall_time_us":50962,"thread_start_us":137280,"threads_started":2}
I20260812 19:07:22.426582 32409 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.428781 32409 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.481168 32409 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:22.484412 30685 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260812 19:07:22.568042 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 100471us
I20260812 19:07:22.573498 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.575678 32558 raft_consensus.cc:348] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:22.583140 32558 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:22.584046 32349 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.584250 32558 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:22.580862 32604 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.587615 32604 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.576165 32502 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.591135 32502 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.594928 32566 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:22.595685 32566 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:22.617331 32502 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:22.621923 32538 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.622959 32604 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:22.622727 32538 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.619151 32536 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.625408 32566 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.625573 32536 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.635742 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:22.636520 29335 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:22.639791 30629 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:22.650624 32644 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:22.651296 32558 ts_tablet_manager.cc:1428] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.588s user 0.014s sys 0.019s
W20260812 19:07:22.651741 32558 ts_tablet_manager.cc:1460] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Tablet startup took 599ms
I20260812 19:07:22.631101 32536 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:22.653836 32180 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.657624 32595 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:22.657841 32538 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:22.656392 32180 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.659962 32180 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.658315 32595 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.667703 32595 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:22.667878 32596 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:22.669168 30674 raft_consensus.cc:1203] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20260812 19:07:22.652132 32558 ts_tablet_manager.cc:1462] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Trace:
0812 19:07:19.272447 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:19.272607 (+ 160us) service_pool.cc:227] Handling call
0812 19:07:19.462972 (+190365us) ts_tablet_manager.cc:928] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
0812 19:07:19.915062 (+452090us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:07:20.082389 (+167327us) tablet_metadata.cc:641] Metadata flushed
0812 19:07:20.102159 (+ 19770us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:07:20.338325 (+236166us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:07:20.341427 (+ 3102us) tablet_replica.cc:205] Creating consensus instance
0812 19:07:20.342087 (+ 660us) ts_tablet_manager.cc:983] Replica registered.
0812 19:07:20.342195 (+ 108us) inbound_call.cc:173] Queueing success response
0812 19:07:20.568989 (+226794us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0812 19:07:22.063725 (+1494736us) ts_tablet_manager.cc:1429] Starting tablet replica
0812 19:07:22.118962 (+ 55237us) tablet_replica.cc:259] Starting instrumentation
0812 19:07:22.323576 (+204614us) tablet_replica.cc:278] Starting consensus
Metrics: {"dns_us":52529,"mutex_wait_us":8805,"raft.queue_time_us":17944,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":9845,"tablet-copy.queue_time_us":148752,"tablet-download-pool.queue_time_us":238,"tablet-download-pool.run_cpu_time_us":2085,"tablet-download-pool.run_wall_time_us":55467,"thread_start_us":305470,"threads_started":2}
I20260812 19:07:22.655205 32586 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260812 19:07:22.669041 32666 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:22.672542 32596 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:22.723518 32626 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.735152 32631 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.661819 32644 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:22.710467 32605 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260812 19:07:22.754256 32644 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:22.762434 32596 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.762727 32547 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.766983 32472 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:22.768225 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:22.724464 32626 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:22.736296 32631 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.746479 32666 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:22.783106 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:22.802835 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.605s user 0.064s sys 0.053s
W20260812 19:07:22.824544 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.627s user 0.065s sys 0.053s
W20260812 19:07:22.827347 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.540s user 0.024s sys 0.056s
W20260812 19:07:22.827836 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.581s user 0.025s sys 0.056s
W20260812 19:07:22.828184 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 581514us
I20260812 19:07:22.833081 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260812 19:07:22.845764 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 46099.7us
I20260812 19:07:22.857434 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:22.870769 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39332
I20260812 19:07:22.871352 29450 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:22.871721 29450 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:22.878722 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.887959 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.889312 32655 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:22.891508 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.899266 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:22.900513 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:22.903977 32643 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260812 19:07:22.845533 30676 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:22.915530 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:22.920153 29601 tablet_copy_service.cc:342] P dd7e8c7a8bc54e73a39411cfc241fe9c: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-8d9f6c46c1a5494aa914b41b3a1616a1 received from {username='slave'} at 127.0.0.1:46208
I20260812 19:07:22.920642 29601 tablet_copy_service.cc:433] P dd7e8c7a8bc54e73a39411cfc241fe9c: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-8d9f6c46c1a5494aa914b41b3a1616a1 on tablet 8d9f6c46c1a5494aa914b41b3a1616a1 with peer a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:22.920789 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 343535us
I20260812 19:07:22.876187 30670 raft_consensus.cc:1203] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260812 19:07:22.928011 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.928200 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.928817 29474 tablet_copy_service.cc:342] P 182784ca1eb74135b8f16d8894fa4b01: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-8cbee78e8845445eb3a4cbba1399e988 received from {username='slave'} at 127.0.0.1:41204
I20260812 19:07:22.929200 29474 tablet_copy_service.cc:433] P 182784ca1eb74135b8f16d8894fa4b01: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-8cbee78e8845445eb3a4cbba1399e988 on tablet 8cbee78e8845445eb3a4cbba1399e988 with peer a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:22.929387 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.935992 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:22.786198 32472 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.785229 32547 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.961836 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:22.968081 32660 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:22.968722 32660 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:22.970993 32472 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.973326 32547 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.975281 32660 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:22.889915 32655 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:22.937711 32626 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.000854 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.001674 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:22.986089 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 413212us
I20260812 19:07:22.954088 32586 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:23.004228 29527 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:23.006239 29450 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.008684 32679 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.019114 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.019378 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.024986 32672 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.025615 32672 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.032068 32631 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:23.009284 32679 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.043102 29578 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33042
I20260812 19:07:23.010898 32605 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.043836 29578 raft_consensus.cc:595] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.020679 32666 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.027864 32643 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:23.082811 32602 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:23.082983 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.083503 29458 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:23.086581 32631 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.096678 32643 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:23.097172 32643 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:23.096899 32631 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.084125 32679 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.085495 32655 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:23.094465 32623 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:07:23.094565 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 44306.7us
I20260812 19:07:23.115407 32631 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.042474 32519 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:23.116533 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.133014 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.147022 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.148062 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:23.153474 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.154140 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.162084 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.163048 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.129383 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:23.123514 29450 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.164274 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.165048 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:23.165802 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:23.193274 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.196456 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.195612 29341 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:23.158150 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.153558 32672 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.202451 32666 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:23.190349 30629 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.209172 32643 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.210222 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.214116 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.112324 32605 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.233629 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.132354 32602 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:23.225534 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 87113.4us
I20260812 19:07:23.243337 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.222882 32648 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:23.144730 32519 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.257246 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:23.262936 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.263049 29403 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.272965 30625 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.276710 32602 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.283593 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.241976 32690 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.289856 32613 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.138567 32623 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:23.235998 32666 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.303690 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.324749 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.327733 29403 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.304023 32667 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:23.304239 32671 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.308481 32672 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:23.311853 32598 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.319002 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.332255 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:23.332415 32682 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.243s user 0.000s sys 0.001s
W20260812 19:07:23.338944 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 96680.2us
I20260812 19:07:23.316324 32690 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.317857 32519 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.344213 32613 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:23.352869 30629 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.355195 32605 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:23.356180 32595 consensus_meta.cc:287] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.685s user 0.006s sys 0.008s
I20260812 19:07:23.356539 32643 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.356680 32595 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.359833 30685 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:23.360631 32595 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:23.363111 32686 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.238s user 0.001s sys 0.000s
I20260812 19:07:23.363736 29569 raft_consensus.cc:3032] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:23.364102 29569 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:23.364737 29569 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.365739 29569 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:23.373471 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 114948us
I20260812 19:07:23.346303 32667 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:23.378093 32667 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:23.389226 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.352411 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:23.368794 32671 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:07:23.394510 32688 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.101s user 0.000s sys 0.001s
I20260812 19:07:23.393636 29281 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.341399 32672 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.400085 29446 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260812 19:07:23.400635 32623 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.402650 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.407028 29569 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:23.408404 30686 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260812 19:07:23.299122 32648 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.409504 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.397714 30676 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:23.420135 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 71453.5us
I20260812 19:07:23.421662 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:23.422612 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:23.423040 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 79906.2us
I20260812 19:07:23.423383 30686 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:23.424276 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.424460 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:23.424674 32673 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.427786 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.425557 32468 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260812 19:07:23.429422 32481 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:23.463661 32658 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:23.469538 30686 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:23.425503 32673 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.432289 32409 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:23.510815 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.528856 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.505300 32673 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:23.536531 32666 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.545866 30671 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:23.538236 32672 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:23.551502 29399 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.465523 32623 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:23.555275 32623 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.538473 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:23.528394 32658 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:23.565570 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 172010us
I20260812 19:07:23.473325 32481 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:23.569808 29281 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.578202 32481 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.581997 32658 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:23.583015 32409 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.584923 32666 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.587215 32644 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:23.590227 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.589470 29281 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.592218 32602 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.594506 29576 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:23.558867 32623 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.598273 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.602010 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:23.602680 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 329794us
I20260812 19:07:23.603266 30625 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.606837 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.607856 29576 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260812 19:07:23.609236 30625 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.611536 32472 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:23.616237 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.617571 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.623910 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:23.617761 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.618654 29459 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:23.626207 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 284499us
I20260812 19:07:23.629231 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:23.620371 32472 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.621083 32631 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.638892 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:23.634613 29530 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.636981 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.639778 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.637885 32566 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260812 19:07:23.642334 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:23.632292 32707 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260812 19:07:23.647553 29527 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:23.612782 32623 consensus_queue.cc:1046] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260812 19:07:23.653362 29530 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:23.637934 32660 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.658898 29281 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.663612 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:23.661520 32502 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.667068 32362 raft_consensus.cc:979] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:23.667495 32362 raft_consensus.cc:1067] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:23.668210 30676 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:23.668875 32547 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.661479 32409 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260812 19:07:23.674124 32519 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:23.677517 32677 raft_consensus.cc:1050] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:23.607451 32648 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:23.652204 32605 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:23.690955 30685 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:23.701797 32712 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:07:23.716604 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 54490.9us
I20260812 19:07:23.717357 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:23.726971 32666 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:23.727686 32666 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:23.730322 32605 raft_consensus.cc:686] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:23.731710 32605 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:23.743393 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 57844.5us
I20260812 19:07:23.728861 32666 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.753129 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:23.751972 32601 consensus_queue.cc:1046] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260812 19:07:23.755736 32623 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.752874 32613 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.793256 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.798801 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.799198 29339 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:23.801476 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:23.804000 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.805298 32690 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.811729 30625 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.814040 29338 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33256
I20260812 19:07:23.814405 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:23.814733 29338 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:23.815222 29338 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:23.816175 32547 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.824180 29338 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:23.826504 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.838593 32666 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:23.757678 30671 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:23.842104 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46222.2us
I20260812 19:07:23.844614 29338 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:23.852777 32655 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.854378 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:23.855021 29459 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:23.864400 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 50078.8us
I20260812 19:07:23.865716 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:23.866302 29579 raft_consensus.cc:3032] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:23.867723 32712 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:23.791448 32566 raft_consensus.cc:1050] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:23.873345 29579 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:23.874181 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:23.872771 32641 raft_consensus.cc:3032] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:23.876247 32641 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:23.681893 32677 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.914572 29579 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:23.915889 29579 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:23.839071 32667 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:23.918690 29451 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:23.920056 29176 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:23.926748 32538 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:07:23.935281 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 61183.4us
I20260812 19:07:23.913844 32566 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.941262 32667 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:23.944979 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 69200.4us
I20260812 19:07:23.947310 29579 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:23.955224 29281 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:23.955866 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:23.957264 29459 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:23.958846 32660 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:23.960736 32660 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:23.962486 32660 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:23.967558 29451 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:23.971864 29177 catalog_manager.cc:5495] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:23.984283 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:23.927424 32641 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:23.994139 32641 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:24.035447 32604 consensus_queue.cc:1046] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:24.037133 32586 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:24.037588 32586 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:24.056459 32586 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:24.061899 32586 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:24.075990 29399 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:24.077735 32667 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:24.165844 29567 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:24.188621 32613 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:24.194929 30679 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260812 19:07:24.208379 30686 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s user 0.048s sys 0.131s
W20260812 19:07:24.208798 30686 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s user 0.048s sys 0.131s
I20260812 19:07:24.213536 32613 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260812 19:07:24.286329 29576 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.010s sys 0.030s
W20260812 19:07:24.287860 29576 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s user 0.010s sys 0.030s
W20260812 19:07:24.297235 32605 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s user 0.017s sys 0.022s
I20260812 19:07:24.333262 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260812 19:07:24.332446 32605 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s user 0.017s sys 0.023s
W20260812 19:07:24.605831 32716 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.340s user 0.001s sys 0.000s
W20260812 19:07:24.635306 32677 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s user 0.049s sys 0.135s
W20260812 19:07:24.641530 32644 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s user 0.005s sys 0.030s
W20260812 19:07:24.654405 32566 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.031s sys 0.036s
W20260812 19:07:24.691324 32566 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s user 0.032s sys 0.036s
W20260812 19:07:24.684099 29176 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.738s user 0.018s sys 0.023s
W20260812 19:07:24.670109 32677 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s user 0.050s sys 0.136s
W20260812 19:07:24.694604 29176 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.748s user 0.018s sys 0.023s
W20260812 19:07:24.678472 32644 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s user 0.005s sys 0.030s
W20260812 19:07:24.706657 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 199173us
I20260812 19:07:24.720343 30686 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 LEARNER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260812 19:07:24.722321 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 113930us
I20260812 19:07:24.764813 29459 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:24.771385 32601 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260812 19:07:24.778257 32724 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.193s user 0.000s sys 0.001s
I20260812 19:07:24.781883 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 1803) took 1097 ms. Trace:
I20260812 19:07:24.790246 29579 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:24.782594 29576 rpcz_store.cc:276] 0812 19:07:23.684527 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:23.684726 (+ 199us) service_pool.cc:227] Handling call
0812 19:07:23.685178 (+ 452us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:23.757656 (+ 72478us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:23.757682 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:23.758657 (+ 975us) log.cc:842] Serialized 564 byte log entry
0812 19:07:24.289174 (+530517us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:24.289211 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:24.327693 (+ 38482us) spinlock_profiling.cc:243] Waited 69.5 ms on lock 0x7b5400253354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:24.327743 (+ 50us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:24.779275 (+451532us) raft_consensus.cc:1671] finished
0812 19:07:24.779298 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:24.781809 (+ 2511us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":159879552,"thread_start_us":530226,"threads_started":1,"wal-append.queue_time_us":592352}
I20260812 19:07:24.767637 32648 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:24.803462 32602 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:24.857800 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:55140 (request call id 1037) took 1153 ms. Trace:
I20260812 19:07:24.858294 29577 rpcz_store.cc:276] 0812 19:07:23.704635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:23.704772 (+ 137us) service_pool.cc:227] Handling call
0812 19:07:24.857359 (+1152587us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400134f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6ed53c6e 00007f9c68456807 00007f9c68456792 00007f9c684566c1 00007f9c6845663c 00007f9c684558f2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:24.857733 (+ 374us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2425695616}
I20260812 19:07:24.866288 32409 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260812 19:07:24.804728 32648 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:24.871989 29179 catalog_manager.cc:5495] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:24.938015 32648 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:24.973584 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:24.987135 29567 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.023603 29453 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:25.026355 30684 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:25.027832 32643 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 19:07:25.029851 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 109974us
W20260812 19:07:25.018867 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 251123us
I20260812 19:07:25.060640 32605 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:24.922108 30670 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.071465 32744 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260812 19:07:25.101287 309 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
I20260812 19:07:25.134745 32744 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:25.135433 32744 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.139199 32744 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:25.148285 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:25.149019 30684 raft_consensus.cc:2379] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:25.155359 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:25.155660 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:25.156746 29445 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:25.208935 29282 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:25.210892 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46608.7us
W20260812 19:07:25.211433 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 47288.7us
I20260812 19:07:25.221647 32409 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:25.222050 32409 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:25.222398 32409 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:25.222236 29282 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:25.234054 32409 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.236068 32409 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:25.238698 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:25.251848 29456 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:25.255676 29282 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:25.258258 32409 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:25.260004 32409 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:25.228974 32602 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:07:25.279613 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 58327.4us
I20260812 19:07:25.270033 32409 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.295835 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:25.410202 317 log.cc:924] Time spent T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.112s user 0.001s sys 0.000s
I20260812 19:07:25.412145 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 367) took 1606 ms. Trace:
I20260812 19:07:25.431535 30671 raft_consensus.cc:2379] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:07:25.468307 320 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260812 19:07:25.481011 32641 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.412576 29176 rpcz_store.cc:276] 0812 19:07:23.805666 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:23.810033 (+ 4367us) service_pool.cc:227] Handling call
0812 19:07:25.412083 (+1602050us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:24.591544 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:24.592044 (+ 500us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:24.592080 (+ 36us) write_op.cc:435] Acquired schema lock
0812 19:07:24.592100 (+ 20us) tablet.cc:660] Decoding operations
0812 19:07:24.592452 (+ 352us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:24.592527 (+ 75us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:24.592562 (+ 35us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:24.592756 (+ 194us) tablet.cc:699] Row locks acquired
0812 19:07:24.592773 (+ 17us) write_op.cc:260] PREPARE: finished
0812 19:07:24.592932 (+ 159us) write_op.cc:270] Start()
0812 19:07:24.593048 (+ 116us) write_op.cc:276] Timestamp: P: 1786561644592878 usec, L: 0
0812 19:07:24.593069 (+ 21us) op_driver.cc:331] REPLICATION: starting
0812 19:07:24.593689 (+ 620us) log.cc:842] Serialized 536 byte log entry
0812 19:07:25.002709 (+409020us) op_driver.cc:444] REPLICATION: finished
0812 19:07:25.029134 (+ 26425us) write_op.cc:301] APPLY: starting
0812 19:07:25.051374 (+ 22240us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:25.063744 (+ 12370us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:25.064644 (+ 900us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:25.092341 (+ 27697us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:25.110963 (+ 18622us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:25.111025 (+ 62us) write_op.cc:312] APPLY: finished
0812 19:07:25.111227 (+ 202us) log.cc:842] Serialized 23 byte log entry
0812 19:07:25.111416 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.111496 (+ 80us) write_op.cc:454] Released schema lock
0812 19:07:25.111532 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:25.111775 (+ 243us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.111801 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":54629,"reported_tablets":1,"spinlock_wait_cycles":17792,"child_traces":[["op",{"apply.queue_time_us":26172,"apply.run_cpu_time_us":2119,"apply.run_wall_time_us":82890,"mutex_wait_us":19663,"num_ops":1,"prepare.queue_time_us":644957,"prepare.run_cpu_time_us":21249,"prepare.run_wall_time_us":115844,"raft.queue_time_us":1034177,"raft.run_cpu_time_us":16685,"raft.run_wall_time_us":493493,"replication_time_us":409544,"spinlock_wait_cycles":36149376,"thread_start_us":935857,"threads_started":4,"wal-append.queue_time_us":272060}]]}
I20260812 19:07:25.577517 29177 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55498 (request call id 361) took 1699 ms. Trace:
I20260812 19:07:25.614125 32739 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:25.615480 32739 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:25.622167 32739 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:25.585400 29177 rpcz_store.cc:276] 0812 19:07:23.878182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:23.878320 (+ 138us) service_pool.cc:227] Handling call
0812 19:07:23.971553 (+ 93233us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b4000000e78. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66ebab7c 00007f9c66eba73b 00007f9c6fd9eb09 00007f9c6fd9eaa0 00007f9c6feda3b2 00007f9c6fd5e0b0 00007f9c6fe634fe 00007f9c6e174fb4 00007f9c6e174f42 00007f9c6e174e71
0812 19:07:25.577182 (+1605629us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:24.762954 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:24.796000 (+ 33046us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:24.798933 (+ 2933us) write_op.cc:435] Acquired schema lock
0812 19:07:24.798968 (+ 35us) tablet.cc:660] Decoding operations
0812 19:07:24.807892 (+ 8924us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:24.807947 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:24.807977 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:24.808183 (+ 206us) tablet.cc:699] Row locks acquired
0812 19:07:24.808196 (+ 13us) write_op.cc:260] PREPARE: finished
0812 19:07:24.808342 (+ 146us) write_op.cc:270] Start()
0812 19:07:24.808437 (+ 95us) write_op.cc:276] Timestamp: P: 1786561644808288 usec, L: 0
0812 19:07:24.808458 (+ 21us) op_driver.cc:331] REPLICATION: starting
0812 19:07:24.809016 (+ 558us) log.cc:842] Serialized 599 byte log entry
0812 19:07:25.478486 (+669470us) op_driver.cc:444] REPLICATION: finished
0812 19:07:25.482166 (+ 3680us) write_op.cc:301] APPLY: starting
0812 19:07:25.482247 (+ 81us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:25.482350 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:25.482467 (+ 117us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:25.482689 (+ 222us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:25.482843 (+ 154us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:25.482887 (+ 44us) write_op.cc:312] APPLY: finished
0812 19:07:25.483082 (+ 195us) log.cc:842] Serialized 23 byte log entry
0812 19:07:25.483283 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.483356 (+ 73us) write_op.cc:454] Released schema lock
0812 19:07:25.483393 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:25.483619 (+ 226us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.483644 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":5297920,"child_traces":[["op",{"apply.queue_time_us":3462,"apply.run_cpu_time_us":1681,"apply.run_wall_time_us":1681,"num_ops":1,"prepare.queue_time_us":768769,"prepare.run_cpu_time_us":2710,"prepare.run_wall_time_us":58457,"replication_time_us":669923}]]}
I20260812 19:07:25.692163 29179 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55520 (request call id 358) took 1081 ms. Trace:
I20260812 19:07:25.733695 334 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:25.734910 334 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.692709 29179 rpcz_store.cc:276] 0812 19:07:24.611032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:24.611173 (+ 141us) service_pool.cc:227] Handling call
0812 19:07:24.750160 (+138987us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b5400002dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6fd68d6c 00007f9c6fe62c3e 00007f9c6e174fb4 00007f9c6e174f42 00007f9c6e174e71 00007f9c6e174dec 00007f9c6e1740a2 00007f9c6824494c 00007f9c68243d86
0812 19:07:25.691890 (+941730us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:24.920497 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:24.932560 (+ 12063us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:24.932591 (+ 31us) write_op.cc:435] Acquired schema lock
0812 19:07:24.932602 (+ 11us) tablet.cc:660] Decoding operations
0812 19:07:24.932915 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:24.932965 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:24.932989 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:24.933173 (+ 184us) tablet.cc:699] Row locks acquired
0812 19:07:24.933187 (+ 14us) write_op.cc:260] PREPARE: finished
0812 19:07:24.933319 (+ 132us) write_op.cc:270] Start()
0812 19:07:24.933430 (+ 111us) write_op.cc:276] Timestamp: P: 1786561644933274 usec, L: 0
0812 19:07:24.933448 (+ 18us) op_driver.cc:331] REPLICATION: starting
0812 19:07:25.060607 (+127159us) log.cc:842] Serialized 532 byte log entry
0812 19:07:25.215884 (+155277us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b5400002dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0a198 00007f9c6d81dade 00007f9c6d81d2b8 00007f9c6d821ba1 00007f9c6d821b59 00007f9c6d821ae9 00007f9c6d821ab1 00007f9c6d820dad 00007f9c7014a204 00007f9c7014a039
0812 19:07:25.478892 (+263008us) op_driver.cc:444] REPLICATION: finished
0812 19:07:25.483966 (+ 5074us) write_op.cc:301] APPLY: starting
0812 19:07:25.484030 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:25.484120 (+ 90us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:25.484139 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:25.484331 (+ 192us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:25.484474 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:25.484516 (+ 42us) write_op.cc:312] APPLY: finished
0812 19:07:25.484672 (+ 156us) log.cc:842] Serialized 23 byte log entry
0812 19:07:25.484788 (+ 116us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.484850 (+ 62us) write_op.cc:454] Released schema lock
0812 19:07:25.484879 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:25.485062 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:25.485091 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41292,"reported_tablets":1,"spinlock_wait_cycles":218693504,"child_traces":[["op",{"apply.queue_time_us":4920,"apply.run_cpu_time_us":1283,"apply.run_wall_time_us":1282,"num_ops":1,"prepare.queue_time_us":2256,"prepare.run_cpu_time_us":15061,"prepare.run_wall_time_us":301325,"replication_time_us":545371,"spinlock_wait_cycles":220311936,"thread_start_us":212519,"threads_started":1}]]}
I20260812 19:07:25.741701 334 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:25.747097 335 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.757516 29339 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.767554 32690 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:25.768836 32707 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:25.768592 32690 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.769560 32707 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:25.785626 32707 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:25.782047 29457 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.807945 32690 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:25.812695 30676 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s user 0.013s sys 0.025s
W20260812 19:07:25.818011 30676 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s user 0.014s sys 0.025s
W20260812 19:07:25.818791 343 log.cc:924] Time spent T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.113s user 0.001s sys 0.001s
I20260812 19:07:25.819639 32690 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:25.820436 32690 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:25.841668 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:25.846786 30686 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:25.871934 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:25.856690 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:25.881455 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 619) took 1068 ms. Trace:
I20260812 19:07:25.879699 29335 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:25.882141 30676 rpcz_store.cc:276] 0812 19:07:24.813489 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:24.848610 (+ 35121us) service_pool.cc:227] Handling call
0812 19:07:24.889021 (+ 40411us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:24.934855 (+ 45834us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:24.934886 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:24.935972 (+ 1086us) log.cc:842] Serialized 564 byte log entry
0812 19:07:25.462910 (+526938us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c66ed45fe 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb
0812 19:07:25.819045 (+356135us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:25.819249 (+ 204us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:25.819410 (+ 161us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:25.881125 (+ 61715us) raft_consensus.cc:1671] finished
0812 19:07:25.881151 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:25.881372 (+ 221us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9978112,"thread_start_us":882219,"threads_started":1,"wal-append.queue_time_us":768777}
I20260812 19:07:25.890245 32690 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:25.918202 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:25.936398 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:25.940215 29578 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:26.062709 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 59056.1us
I20260812 19:07:25.897539 29338 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:26.090763 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.091722 32762 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:26.092494 32762 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.094357 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.096140 32762 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:26.105118 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:26.106741 29335 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:26.122941 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.124276 29446 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:07:26.181475 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 115047us
I20260812 19:07:26.179468 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:26.194267 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 58480us
I20260812 19:07:26.207508 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:26.207842 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.209719 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:26.217357 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 85993us
I20260812 19:07:26.200904 32762 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260812 19:07:26.218353 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 87850.2us
I20260812 19:07:26.220618 32762 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:26.223974 334 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:26.235088 32762 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.237659 334 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:26.249327 334 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.264358 340 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:26.264925 340 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:26.268505 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:26.270758 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 66416.3us
W20260812 19:07:26.304499 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 80323.5us
W20260812 19:07:26.306331 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 82801.6us
I20260812 19:07:26.309465 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:26.338111 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 105806us
I20260812 19:07:26.347080 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:26.357451 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49033.7us
I20260812 19:07:26.326428 29445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:26.398856 29445 raft_consensus.cc:595] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:26.430399 375 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:26.431864 30684 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:26.432541 338 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:26.433085 338 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:26.433493 338 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:26.440766 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 80710.3us
I20260812 19:07:26.442876 340 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:26.418430 30671 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:26.473224 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 69018.3us
I20260812 19:07:26.482352 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:26.492130 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:26.495244 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:26.496049 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260812 19:07:26.605043 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 94405.9us
I20260812 19:07:26.499730 29174 catalog_manager.cc:5495] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 3, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:26.612351 29174 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:26.620645 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 57155.8us
I20260812 19:07:26.628141 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.629508 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:26.633198 32762 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:26.639523 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 111239us
I20260812 19:07:26.619309 29174 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 4 to 5, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:26.641028 30629 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:26.505885 334 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.647872 29174 catalog_manager.cc:5495] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:26.648913 334 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:26.651189 389 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:26.651538 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:26.652845 379 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:26.673987 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:26.676218 29459 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:26.676772 32762 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:26.675812 29576 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:26.680483 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:26.681238 29578 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:07:26.685403 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 48245.5us
I20260812 19:07:26.687892 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:26.697394 368 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:26.693039 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:26.701465 32751 raft_consensus.cc:979] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:26.702044 32751 raft_consensus.cc:1067] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:26.649472 338 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.709337 29446 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:26.723968 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:26.724866 29578 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:26.730329 29399 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:26.731379 339 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260812 19:07:26.755579 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 113992us
I20260812 19:07:26.759640 338 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:26.760304 29338 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.767460 29337 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:26.768134 29337 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:26.768496 29337 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.775969 29338 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:26.775523 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:26.788460 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:26.789417 29567 raft_consensus.cc:2354] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:26.797907 335 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:26.799790 335 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } attrs { replace: true } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:26.811172 29337 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.811244 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:26.812757 32739 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:26.813268 32739 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:26.823345 29281 leader_election.cc:398] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:26.823946 29281 leader_election.cc:401] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260812 19:07:26.828701 32692 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:26.831331 29337 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:26.836265 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:26.837046 29446 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:26.838762 29446 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:26.843192 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:26.843904 29567 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.814226 32739 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.851894 335 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:26.856423 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:26.858397 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:26.839861 29446 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.871608 29527 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:26.880467 370 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:26.880767 32736 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:26.884431 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:26.912065 29567 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:26.921247 29281 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:26.922546 369 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:26.923062 369 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:26.923897 369 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:26.926144 29446 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:26.953841 29446 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:27.002620 29338 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:27.014309 29177 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 2 to 3, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:27.023046 29458 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:27.025173 30671 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:27.028306 32648 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:27.030212 32648 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:27.069752 387 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.071444 30671 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.094920 29459 raft_consensus.cc:2932] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.143144 29179 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 4 to 5, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 5 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:27.150646 335 consensus_queue.cc:1159] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "182784ca1eb74135b8f16d8894fa4b01" responder_term: 1 status { last_received { term: 1 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 1 index: 3 } }
I20260812 19:07:27.151880 29576 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:27.157336 29567 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:27.178927 30670 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.181087 334 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:27.180768 29176 catalog_manager.cc:5495] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:27.196360 29337 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.204592 361 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:27.217725 29151 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet f55b37aa5102423ebbccef3f872d85e8 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:27.240053 30684 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.243644 361 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:27.253686 391 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.260955 29177 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 5 to 6, VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) evicted. New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.269804 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:27.271162 29339 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.280002 29179 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:27.283209 29459 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:27.304641 29458 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.308881 29558 tablet_service.cc:1514] Processing DeleteTablet for tablet f55b37aa5102423ebbccef3f872d85e8 with delete_type TABLET_DATA_TOMBSTONED (TS dd7e8c7a8bc54e73a39411cfc241fe9c not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:55140
I20260812 19:07:27.308972 335 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260812 19:07:27.337039 29459 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:27.339727 32692 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:27.340262 29576 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.346570 381 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260812 19:07:27.382686 29339 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:27.370630 30676 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260812 19:07:27.386464 335 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:27.397177 32751 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:27.406569 381 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.401108 30671 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.411394 29580 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.398002 335 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.426707 30676 raft_consensus.cc:2932] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.445660 428 tablet_replica.cc:330] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: stopping tablet replica
I20260812 19:07:27.447960 29456 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:27.449424 29578 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:27.450613 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d9f6c46c1a5494aa914b41b3a1616a1 with delete_type TABLET_DATA_TOMBSTONED (TS 182784ca1eb74135b8f16d8894fa4b01 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:56850
I20260812 19:07:27.451912 391 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:27.469668 30684 raft_consensus.cc:2932] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.465371 365 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260812 19:07:27.494585 436 tablet_replica.cc:330] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: stopping tablet replica
I20260812 19:07:27.495800 436 raft_consensus.cc:2229] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:27.496474 436 raft_consensus.cc:2258] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:27.515553 29458 raft_consensus.cc:2932] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.524777 29148 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8cbee78e8845445eb3a4cbba1399e988 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:27.509074 29177 catalog_manager.cc:5495] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.554111 428 raft_consensus.cc:2229] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:27.555291 428 raft_consensus.cc:2258] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:27.562947 428 ts_tablet_manager.cc:1905] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:27.564270 436 ts_tablet_manager.cc:1905] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:27.570411 29146 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet f6dae5f1cee3498787a4a28802d59d16 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 19:07:27.589224 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cbee78e8845445eb3a4cbba1399e988 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:27.720835 443 tablet_replica.cc:330] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:27.765918 29176 catalog_manager.cc:5495] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df reported cstate change: config changed from index 3 to 4, VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) evicted. New cstate: current_term: 1 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:27.792256 436 ts_tablet_manager.cc:1918] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:27.793414 436 log.cc:1196] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 182784ca1eb74135b8f16d8894fa4b01: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-1-root/wals/8d9f6c46c1a5494aa914b41b3a1616a1
I20260812 19:07:27.797142 29148 catalog_manager.cc:4841] TS 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): tablet 8d9f6c46c1a5494aa914b41b3a1616a1 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:27.816694 443 raft_consensus.cc:2229] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:27.817653 443 raft_consensus.cc:2258] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:27.823005 443 ts_tablet_manager.cc:1905] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:27.829381 29314 tablet_service.cc:1514] Processing DeleteTablet for tablet f6dae5f1cee3498787a4a28802d59d16 with delete_type TABLET_DATA_TOMBSTONED (TS 6dd313c6592d4f33bab314a73f28b007 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37750
I20260812 19:07:27.831904 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:27.850574 428 ts_tablet_manager.cc:1918] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260812 19:07:27.851238 428 log.cc:1196] T f55b37aa5102423ebbccef3f872d85e8 P dd7e8c7a8bc54e73a39411cfc241fe9c: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-2-root/wals/f55b37aa5102423ebbccef3f872d85e8
I20260812 19:07:27.860950 29146 catalog_manager.cc:4841] TS dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573): tablet f55b37aa5102423ebbccef3f872d85e8 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
W20260812 19:07:27.864838 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 87215.4us
I20260812 19:07:27.878368 443 ts_tablet_manager.cc:1918] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:27.878885 443 log.cc:1196] T 8cbee78e8845445eb3a4cbba1399e988 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/8cbee78e8845445eb3a4cbba1399e988
I20260812 19:07:27.880653 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet 8cbee78e8845445eb3a4cbba1399e988 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:27.882476 443 tablet_replica.cc:330] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: stopping tablet replica
I20260812 19:07:27.885015 443 raft_consensus.cc:2229] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 19:07:27.885918 443 raft_consensus.cc:2258] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 19:07:27.888931 443 ts_tablet_manager.cc:1905] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 19:07:27.909562 443 ts_tablet_manager.cc:1918] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260812 19:07:27.910125 443 log.cc:1196] T f6dae5f1cee3498787a4a28802d59d16 P 6dd313c6592d4f33bab314a73f28b007: Deleting WAL directory at /tmp/dist-test-taskw107rr/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786561513794812-27445-0/minicluster-data/ts-0-root/wals/f6dae5f1cee3498787a4a28802d59d16
I20260812 19:07:27.912806 29151 catalog_manager.cc:4841] TS 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): tablet f6dae5f1cee3498787a4a28802d59d16 (table test-workload [id=ed044727589a44c1a5f2e214f86101ed]) successfully deleted
I20260812 19:07:27.918728 29339 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "647dfc8bad934787a924a06988dda474"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:27.919245 29339 raft_consensus.cc:595] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:27.921728 29339 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:27.922256 29339 raft_consensus.cc:595] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:27.936822 29339 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:27.937433 29339 raft_consensus.cc:595] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:27.942361 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:27.943919 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:27.944586 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:28.192240 32692 raft_consensus.cc:979] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:28.192817 32692 raft_consensus.cc:1067] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:28.194568 30676 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "647dfc8bad934787a924a06988dda474"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:28.195058 30676 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:28.195338 30676 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:28.200911 30676 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:28.203262 30676 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:28.203416 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:28.204022 29339 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:28.204387 29339 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:28.204253 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:28.204942 29459 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:28.205247 29339 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:28.206439 29339 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:28.212852 29459 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:28.215477 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:28.216506 453 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:28.216799 29339 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:28.217002 453 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:28.217979 453 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:28.228922 29177 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:28.385077 369 raft_consensus.cc:979] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:28.385543 365 raft_consensus.cc:1067] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
W20260812 19:07:29.251113 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.538s user 0.000s sys 0.008s
W20260812 19:07:29.251675 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.538s user 0.000s sys 0.009s
W20260812 19:07:29.252455 30626 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.005s sys 0.003s
W20260812 19:07:29.252938 30626 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.507s user 0.005s sys 0.003s
W20260812 19:07:29.258818 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.003s sys 0.010s
W20260812 19:07:29.259323 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.543s user 0.003s sys 0.010s
I20260812 19:07:29.399124 455 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.399955 455 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.402544 455 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.502056 455 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.502828 455 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.503674 462 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.504357 462 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.507269 462 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.507865 462 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.508412 462 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.510970 462 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.517874 455 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.537742 456 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.538523 456 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.541427 456 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:29.542048 456 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.543015 456 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.544639 470 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.549350 470 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.551549 470 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.569898 466 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.570569 466 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.559219 465 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.547098 462 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.566592 453 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.564088 464 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.568001 475 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.553831 456 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:29.572899 465 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.557761 455 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.597020 456 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.627712 478 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.628459 492 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.628434 478 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.629073 492 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.580598 462 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.633847 446 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.580978 453 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.582432 464 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.625068 455 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.645084 478 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.649729 480 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.650635 465 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.651465 489 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.638327 446 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.654603 480 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.632155 456 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.642267 474 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.652748 484 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.682228 446 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.682139 484 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.682812 453 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.683349 453 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.683797 453 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.684248 484 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.676839 509 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.659533 491 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.678598 462 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.595057 475 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.677917 507 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.696938 484 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.685858 474 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.697623 484 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.698787 509 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.710646 509 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.702363 478 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.680312 489 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.717693 478 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.719048 480 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.691053 492 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.719894 478 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.709781 484 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.720516 455 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.721308 466 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.700887 491 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.722190 474 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.722769 477 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.723368 477 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.700309 507 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.725613 477 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.723829 491 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.726137 507 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.717669 502 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.727480 464 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.727170 502 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.729538 490 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.742712 465 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.764144 511 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.761332 490 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.769924 475 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.769785 511 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.770794 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:29.771674 490 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:29.755340 464 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.776582 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:29.771754 511 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.764068 503 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.781358 464 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.782375 503 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.784200 464 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.758255 497 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.784442 503 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.756327 510 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.784869 497 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.785758 510 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.787144 497 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.778837 512 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.787482 29332 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:29.768985 465 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.787896 512 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.759694 504 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.791059 504 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.793413 504 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.760890 513 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:29.795723 513 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.769129 489 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.766122 493 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.798749 493 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.800835 493 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.780761 502 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.814575 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:29.815344 29338 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:29.816519 470 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:29.817183 470 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:29.819360 470 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.824812 515 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:29.825488 515 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.827678 515 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.781446 29457 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:29.781342 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:29.775682 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:29.781819 456 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:29.778090 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:29.789430 465 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.108871 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:29.858628 497 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.133033 497 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.109234 517 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.134142 517 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.135852 497 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.136269 517 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.109880 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.103772 521 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.138334 521 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.140448 521 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.101130 515 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.141213 515 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.142177 29326 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:30.143097 515 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.120170 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.145152 29332 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:07:29.847900 504 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.106057 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.123096 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.146400 504 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.105870 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:29.833846 511 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.123805 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.124467 477 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.868093 470 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.147787 477 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.148432 470 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.149824 477 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.150628 470 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.843511 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.125065 491 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.852195 490 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:29.867633 466 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.151837 491 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.152388 469 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:30.152187 466 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.018281 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:29.831718 464 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.152868 469 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.153244 464 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.153929 491 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.155076 29458 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:29.843851 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.131068 502 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.156073 29445 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:30.156126 502 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.157023 466 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.157528 466 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.158262 502 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.158027 466 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.132303 29403 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.161559 29403 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.123368 512 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.165207 512 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.166147 512 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.166627 512 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.147650 511 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.169426 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.170028 511 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:29.843281 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:29.787899 510 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:29.883287 475 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.174248 475 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.176407 475 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.122370 513 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.162186 491 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:30.133458 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.295926 491 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.296121 29341 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:30.183966 477 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.277144 512 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.296865 477 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.185534 521 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.298204 521 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.298700 29403 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.299528 475 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.299897 475 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.299991 521 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.185606 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.186370 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.190092 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.191411 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.300297 475 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.193272 465 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.306456 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.306975 465 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.192536 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.228652 29450 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:30.230384 497 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.155463 464 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.235088 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.265058 504 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.310473 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.310750 464 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.256132 511 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.311398 464 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.268277 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.312108 511 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.313092 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.313318 464 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.265156 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.314082 511 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.314581 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.309520 497 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.315552 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.315709 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.269124 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.278105 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.277129 466 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.290431 470 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.316838 497 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.283069 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.180651 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.317991 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.317263 512 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.297231 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.319224 29341 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:30.186302 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.292019 526 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.301286 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.305918 477 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.155555 490 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.320417 29334 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.317317 470 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.321787 490 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.296418 491 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:30.322322 470 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.317353 497 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.323159 497 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.309443 29450 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:30.324919 29450 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.317217 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.325018 497 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.318588 29445 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:30.308969 465 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.300645 528 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.311769 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.326400 29451 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:30.326225 528 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.326705 490 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.327200 490 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.329016 490 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.329473 490 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.329895 490 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.330878 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.331696 490 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.331736 528 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.332747 470 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:30.333098 497 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.333403 470 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.333936 470 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:30.334374 529 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.334877 529 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.338151 511 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.338855 29282 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.333639 497 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.339051 511 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.340891 511 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.338505 528 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.341598 528 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.340799 497 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.343482 528 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.345268 529 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.357366 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.355371 490 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.358340 490 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.345582 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:30.346866 513 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:30.359350 513 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.359764 513 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:30.360301 491 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:29.682768 446 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.361884 446 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.362766 491 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.365828 475 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.366784 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.367092 446 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.367570 446 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.367897 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.368194 475 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.367985 446 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.368677 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.369720 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.370267 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.373710 513 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.374253 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.374828 529 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:30.375219 529 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.375308 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.375589 529 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:30.370553 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.347013 470 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.306536 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.349210 526 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.370710 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.337446 29450 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.371673 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.371914 456 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.379487 456 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.380553 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.381319 456 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.381850 29458 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.381842 456 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.382066 470 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.382570 456 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.382539 490 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.383180 529 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:30.350935 30626 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.669s user 0.013s sys 0.029s
I20260812 19:07:30.316788 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.352114 511 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.384354 511 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.369915 446 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.354022 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.379176 29450 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.354784 525 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.386297 511 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.355346 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.355784 453 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.387066 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.387393 526 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.387363 453 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.388132 453 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.358637 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.345209 29334 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:30.390039 29332 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.353663 466 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.351501 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.352699 29457 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.390735 466 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.353686 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.385805 29445 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:30.383428 513 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.384851 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.392562 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.391665 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:30.393244 29578 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:30.386518 525 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.392589 453 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.381603 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.394472 453 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.395120 525 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.383370 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.394927 453 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.401059 453 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.403455 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.403620 540 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.403905 453 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.395864 466 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.397127 515 consensus_queue.cc:1046] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:30.404184 540 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.404340 453 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.405563 515 raft_consensus.cc:3032] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:30.405938 515 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:30.395771 446 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.356279 528 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.401234 29530 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.402032 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.403143 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.406785 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.406354 533 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.412007 29456 raft_consensus.cc:2379] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 2: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:30.406927 515 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.412333 533 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.415093 515 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.407054 469 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.407451 446 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.418844 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 515) took 2028 ms. Trace:
I20260812 19:07:30.419767 29459 rpcz_store.cc:276] 0812 19:07:28.390857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:29.716718 (+1325861us) service_pool.cc:227] Handling call
0812 19:07:29.717041 (+ 323us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:29.728626 (+ 11585us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:29.728660 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:29.728699 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:29.728730 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:30.418599 (+689869us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54003bd654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:30.418647 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:30.418764 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":26172160}
I20260812 19:07:30.406306 540 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.421386 540 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.422094 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.427496 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.450922 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.452071 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.453169 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.428277 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:30.430007 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.785s user 0.016s sys 0.021s
I20260812 19:07:30.434022 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.434378 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.435129 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.436213 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.437306 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.437991 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.438131 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.439486 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.440662 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.441694 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.442687 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.442425 29331 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:30.460777 30684 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:30.461285 502 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.461678 502 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.462703 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.463053 529 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.463712 502 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.463934 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.464735 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.465021 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.465564 29572 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:30.466653 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.466805 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.467561 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.467854 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.468596 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.468854 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.466712 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.460232 29580 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:30.471386 540 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.408843 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:30.478497 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:30.479058 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 70289.8us
I20260812 19:07:30.479347 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.480020 540 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.408802 453 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.445662 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.409744 511 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.481324 511 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.481809 511 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.481374 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.479069 29580 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:07:30.410383 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.410944 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.482880 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.411216 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.483109 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:30.483315 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.447229 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.483906 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.484220 30670 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:30.484344 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.409253 29450 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:07:30.409323 29458 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:07:30.485023 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.446564 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.485473 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.408860 528 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.492650 29574 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:07:30.493057 528 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.410665 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.493352 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.355816 464 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.430053 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:30.460186 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.817s user 0.022s sys 0.022s
I20260812 19:07:30.494306 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.496417 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.496603 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.497464 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.497911 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.499120 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.499565 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.499703 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.500260 29576 raft_consensus.cc:2379] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:30.500787 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.501000 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.501802 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.501996 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.502389 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.502992 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.503404 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.501856 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.503978 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.473456 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.475438 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.505015 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.505321 543 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.505899 543 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.443323 541 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.507153 541 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.508010 543 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.509120 541 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.476946 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.510262 30671 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:07:30.465842 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.477735 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.477864 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.482187 511 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.513386 464 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.514659 29575 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:30.515726 464 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.485703 30686 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:07:30.384857 456 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.407617 466 consensus_queue.cc:1046] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260812 19:07:30.517421 466 raft_consensus.cc:3032] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:30.500268 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.517791 466 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:30.518795 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.518625 466 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.519797 466 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.451036 446 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.485601 29580 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:07:30.487385 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:30.490099 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:30.490208 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.480418 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.523531 528 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.503468 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.523936 528 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.504117 29334 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:30.524360 528 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.505125 29457 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:30.475456 494 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:30.477080 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.522964 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:30.525305 494 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.521817 511 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.527853 511 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.534911 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.535717 470 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.536078 470 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.536422 470 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.537392 456 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.505020 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.493436 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.549098 29574 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.421809 533 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:30.557308 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 66238.6us
I20260812 19:07:30.563295 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.564016 517 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.564067 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.565088 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.565740 517 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:30.565996 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.566643 30686 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:30.566753 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.567621 517 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.567687 494 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.568065 517 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.568447 517 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.538012 456 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:30.571278 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 81243.3us
I20260812 19:07:30.571359 533 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:30.571884 456 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.572001 533 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.572625 533 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:07:30.573385 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 50523.8us
I20260812 19:07:30.575064 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.575634 464 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.576053 464 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.576476 464 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.591944 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.591742 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.582032 469 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.580379 517 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.587321 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.572121 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:30.594010 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.597054 475 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.597373 547 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.597463 475 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.597975 475 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.598675 456 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:30.598093 542 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.600481 542 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.600818 542 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.596699 469 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.603994 469 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.604288 517 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.605161 29403 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.605899 491 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:30.606381 491 raft_consensus.cc:686] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:30.607219 491 consensus_queue.cc:237] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.609171 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.609484 29403 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.610270 465 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.610667 465 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.610589 546 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:30.611164 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.612349 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.611129 465 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.613163 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.614806 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.615350 29341 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:30.616632 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.616781 30629 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.618624 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.618630 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.619848 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.620143 30671 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:30.621373 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.622903 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.586984 528 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.610386 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.625301 535 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.625842 464 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.612287 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.627363 29580 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:30.636386 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.637039 30670 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:30.638244 29576 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.627547 465 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.639351 542 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.629421 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:30.630106 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.640307 29458 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:30.630215 30629 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.632082 29569 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260812 19:07:30.633176 29281 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.68:41925 timed out after 1.534s (SENT) [suppressed 1 similar messages]
I20260812 19:07:30.632313 475 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.641599 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.635166 533 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.634132 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.633839 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.634662 528 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.645648 475 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.647708 533 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.648360 29574 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:30.649652 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.645617 465 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.650756 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.651983 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.652638 30670 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:07:30.652925 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.653520 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.654098 30684 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
W20260812 19:07:30.650791 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 59127.1us
I20260812 19:07:30.655540 29403 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.655900 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.656046 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.656107 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.656826 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.628237 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.657856 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.581610 470 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.641721 456 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.642136 30629 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.661904 29457 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:30.665246 512 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.665668 512 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:30.666445 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.667136 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.661175 29282 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.628882 29530 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.644425 29450 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.668403 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.665738 456 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.641088 542 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.666127 477 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.666601 512 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.669255 29530 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.669579 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:30.642215 464 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.670604 464 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.671240 464 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.671581 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.671748 29530 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.672690 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.673097 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.685869 477 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.684819 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.685132 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:30.684958 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 51596us
I20260812 19:07:30.687687 29399 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.635710 29331 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:30.675606 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.676019 515 consensus_queue.cc:1046] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:30.676085 547 consensus_queue.cc:1046] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260812 19:07:30.673035 29530 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.676977 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.696507 29399 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.678032 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.677651 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.679076 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.696462 547 raft_consensus.cc:3032] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
W20260812 19:07:30.684942 29619 debug-util.cc:398] Leaking SignalData structure 0x7b080014c580 after lost signal to thread 27450
I20260812 19:07:30.698024 547 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:30.699489 29403 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.699793 29530 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
W20260812 19:07:30.698951 29619 debug-util.cc:398] Leaking SignalData structure 0x7b08002000c0 after lost signal to thread 29236
I20260812 19:07:30.700079 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.698868 547 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:07:30.701319 29619 debug-util.cc:398] Leaking SignalData structure 0x7b08003303c0 after lost signal to thread 29379
I20260812 19:07:30.701692 547 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.701927 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:30.702605 29619 debug-util.cc:398] Leaking SignalData structure 0x7b080035ae00 after lost signal to thread 29496
I20260812 19:07:30.703523 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.704197 491 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:30.704720 491 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:30.705029 511 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.705085 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.689872 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.705601 491 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.691037 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.706879 29530 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.685102 470 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.708206 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.696398 515 raft_consensus.cc:3032] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:30.708866 515 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:30.708935 528 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:07:30.709687 515 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:30.703717 29619 debug-util.cc:398] Leaking SignalData structure 0x7b080037a6e0 after lost signal to thread 29622
I20260812 19:07:30.683035 512 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.710914 469 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.715332 515 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.715692 29446 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.673245 464 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.695631 477 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:30.717880 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.718678 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.719401 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.708506 529 raft_consensus.cc:2741] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260812 19:07:30.712306 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.710249 517 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:30.694891 29527 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.721264 29527 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.721673 529 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.682869 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.714215 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.721596 517 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:30.726385 529 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:30.719580 513 consensus_queue.cc:1046] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:30.719023 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:30.726765 529 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:30.726977 29619 debug-util.cc:398] Leaking SignalData structure 0x7b08002f56c0 after lost signal to thread 30721
I20260812 19:07:30.726692 517 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.713626 512 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.726785 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.728638 529 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:30.727303 29619 debug-util.cc:398] Leaking SignalData structure 0x7b0800350100 after lost signal to thread 30724
I20260812 19:07:30.732236 29527 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.735726 529 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.734722 494 raft_consensus.cc:3032] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:30.740907 494 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:30.741845 494 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.743058 494 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.745241 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.746404 529 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.736784 511 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.734015 517 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.738137 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.737701 477 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:30.752717 29446 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:30.753571 30623 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.754338 533 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.714119 470 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260812 19:07:30.739689 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 44463.4us
I20260812 19:07:30.755041 547 consensus_queue.cc:1046] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:30.756042 547 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:30.754745 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.756369 547 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:30.756943 533 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.756161 517 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.757151 547 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.747637 512 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.758760 547 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:30.755208 29281 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.757056 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:30.760202 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.760531 29446 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:30.747582 29399 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.736300 29527 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.762778 29527 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.762861 464 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.758421 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.763782 528 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.764397 29337 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:30.764791 549 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.764998 29527 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.766116 29527 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.766310 549 consensus_queue.cc:1046] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:30.766044 30623 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.767352 542 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.767304 29527 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.767840 30623 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.768709 29527 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.768821 512 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.769497 466 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.768890 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.770239 466 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.770339 533 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.770630 469 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.767894 528 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.771756 29527 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.771922 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.772466 30623 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.774020 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.780902 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.772289 529 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:30.784647 529 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.785032 529 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:30.789278 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.777310 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.769496 515 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.778509 513 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.779599 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.791201 29337 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:30.780189 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.780634 547 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.781042 464 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:30.780740 29527 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.776003 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:30.791997 547 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:30.792495 547 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.783977 533 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.793000 29527 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.793084 29403 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:30.794137 29527 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.791494 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.795181 29527 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.795413 30623 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.795966 29527 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.796587 30623 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.796792 29527 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.797602 29527 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.793929 512 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:30.798658 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.800940 542 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.802153 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:30.799474 525 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.804042 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.817902 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "59710c65553f42188b78bc9bc0324542"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:30.818728 29337 raft_consensus.cc:595] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.802917 542 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:30.800082 30623 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.820523 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.799335 466 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.824265 512 raft_consensus.cc:686] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:30.825531 512 consensus_queue.cc:237] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.821817 542 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:30.801573 528 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.826874 542 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:30.827015 528 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.827484 528 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.799031 491 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.799252 490 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.801035 494 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.823360 29177 catalog_manager.cc:5495] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:30.822068 513 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.830652 29331 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:30.800997 526 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.792083 464 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.799324 524 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.801144 549 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.799397 497 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.800932 470 raft_consensus.cc:2741] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260812 19:07:30.842540 30623 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.845276 29569 raft_consensus.cc:1226] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:30.847364 541 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.848760 30623 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.849526 541 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.850670 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.851526 541 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.851908 541 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.852278 541 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.856205 30623 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.857023 533 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.857393 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:30.863150 30623 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.863806 533 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:30.864235 533 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.864097 470 consensus_queue.cc:1046] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:30.864634 533 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.845492 456 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:30.867694 456 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:30.868590 456 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.868815 542 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.870878 529 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.872917 529 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.872761 533 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.873607 541 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.877707 546 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.878783 30623 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.880133 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.881192 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.842762 464 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:30.841459 526 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:30.869877 456 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.898329 29569 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:30.898200 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.845099 446 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:07:30.900585 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 63849.6us
I20260812 19:07:30.903167 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.903970 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.892454 543 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.910991 542 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:30.894611 549 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.912436 29576 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 2: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:30.899353 526 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:30.902740 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.923154 528 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.925438 528 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.926656 526 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.908041 533 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.929515 526 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.933331 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.934983 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.935935 29527 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.936301 477 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.908865 541 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:30.936314 562 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.938387 526 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.940774 469 raft_consensus.cc:3032] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:30.941810 469 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:30.949520 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.939344 558 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.949676 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.951400 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.951999 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.952508 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:30.952661 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:30.960328 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.962580 29337 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:30.964526 470 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.965571 29326 raft_consensus.cc:1226] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:30.966084 469 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.965703 502 raft_consensus.cc:979] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:30.967496 469 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:30.967869 502 raft_consensus.cc:1067] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Signalling peer 182784ca1eb74135b8f16d8894fa4b01 to start an election
I20260812 19:07:30.968403 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.964542 29337 raft_consensus.cc:595] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.969233 562 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.966564 512 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.971704 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.972950 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.973673 512 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:30.974078 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.974289 512 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:30.975392 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.975600 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:30.976820 562 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.977748 464 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.978549 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.979759 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:30.980382 470 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.980867 29399 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.980999 470 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.981676 470 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.982054 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.967970 547 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:30.983752 512 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:30.983999 547 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:30.985170 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:30.986423 470 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:30.987807 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.987725 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.989334 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.990345 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:30.994058 529 consensus_queue.cc:1046] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:30.995121 529 raft_consensus.cc:3032] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:30.995425 529 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:30.996238 529 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:30.997395 529 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:30.999032 464 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:31.005043 29458 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "59710c65553f42188b78bc9bc0324542"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:39326
I20260812 19:07:31.005662 29458 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:31.006026 29458 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.006126 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.006711 29450 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260812 19:07:31.008047 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.010082 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:31.015678 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:31.016960 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260812 19:07:31.018668 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:31.020502 29458 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:31.022476 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 185519us
I20260812 19:07:31.025218 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.026101 515 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.026925 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.027179 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.027525 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.028884 29458 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:31.027204 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.030295 29459 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:31.030777 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:31.033358 29331 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:31.033653 29331 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.031524 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:31.033928 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.035101 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.034888 29331 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.036015 29331 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.036187 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.037707 29445 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:31.038923 29451 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:31.031926 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.040000 29446 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:31.041880 29459 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:31.043551 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.045114 564 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.045308 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.044417 29281 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.047935 564 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.049039 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.051851 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:31.055923 29331 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:31.056413 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.054160 29281 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.057511 29337 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.058585 564 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.059885 29281 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.062500 564 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.060696 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.064348 29574 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:31.064780 29576 raft_consensus.cc:1226] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:31.061367 29403 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:31.066919 512 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:31.067214 469 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:31.067440 512 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.068223 446 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.068405 512 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.069242 30625 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.070994 446 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.070914 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:31.071681 29580 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.070432 30625 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.073254 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.074718 30684 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:31.075565 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.077936 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:31.078593 29528 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.079463 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.081313 30625 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.075538 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.077827 541 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.083240 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:31.083698 563 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.083938 29569 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:31.080214 535 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.085898 446 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:31.078473 456 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.088917 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.089763 563 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.090067 541 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.089823 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.091477 464 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.091535 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.092103 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.089107 470 consensus_queue.cc:1046] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260812 19:07:31.092965 563 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.093519 470 raft_consensus.cc:3032] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:31.093178 558 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.093914 470 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.092886 29528 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.094616 562 consensus_queue.cc:1046] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:31.094825 470 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.095583 558 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.096064 563 raft_consensus.cc:3032] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:31.096453 563 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.096196 558 consensus_queue.cc:1046] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:31.098965 29580 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:31.089696 529 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.100189 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.103173 528 consensus_queue.cc:1046] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:31.104199 558 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:31.104598 558 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:31.105645 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.105350 558 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.108428 29580 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:31.111800 30625 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.111248 469 consensus_queue.cc:1046] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260812 19:07:31.115180 528 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:31.115585 528 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:31.097201 563 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.116364 528 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.116956 570 consensus_queue.cc:1046] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260812 19:07:31.117210 563 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:31.117269 494 consensus_queue.cc:1046] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:31.119174 30625 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.120393 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:31.096230 470 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.133307 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.125142 453 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:31.124835 541 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.113101 477 consensus_queue.cc:1046] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260812 19:07:31.137943 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.129678 529 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:31.142306 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.132331 570 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.129262 446 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.134642 547 raft_consensus.cc:3032] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:31.141273 30684 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:31.206933 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 98822.7us
I20260812 19:07:31.218122 531 consensus_queue.cc:1046] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260812 19:07:31.141270 477 raft_consensus.cc:3032] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:31.136973 475 consensus_queue.cc:1046] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260812 19:07:31.206128 525 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.212435 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:31.213600 30681 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:31.145394 547 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.129841 533 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260812 19:07:31.136574 558 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:31.138330 497 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:31.144119 526 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:31.202502 528 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:31.242537 491 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:31.258348 491 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.258612 547 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.259714 491 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.260775 547 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:31.261294 491 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.245224 464 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:31.266968 29580 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.244767 477 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.265054 546 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:31.289527 577 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:31.289736 465 raft_consensus.cc:3032] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:31.290740 577 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.284273 546 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:31.291630 577 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.291817 546 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.293056 577 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.293151 546 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.290833 465 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.282783 475 consensus_queue.cc:1046] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260812 19:07:31.319092 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 45884.2us
I20260812 19:07:31.316645 477 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.280469 512 consensus_queue.cc:1046] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260812 19:07:31.327754 475 raft_consensus.cc:3032] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:31.341114 29459 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:31.343335 30681 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
W20260812 19:07:31.343956 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 66960.4us
I20260812 19:07:31.359790 29337 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:31.362474 589 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260812 19:07:31.343436 475 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:31.376583 588 consensus_queue.cc:1046] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260812 19:07:31.382124 465 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.383615 465 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:31.383767 475 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.385092 475 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:31.370317 477 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:31.386880 525 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:31.350591 490 consensus_queue.cc:1046] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260812 19:07:31.387285 525 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:31.377841 588 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:31.387733 536 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260812 19:07:31.394362 30671 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:31.392864 588 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.409523 29337 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.409879 29341 raft_consensus.cc:1261] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.403553 565 raft_consensus.cc:979] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:31.403529 588 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.407374 536 consensus_queue.cc:1046] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260812 19:07:31.443789 29576 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.427889 491 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260812 19:07:31.449288 578 raft_consensus.cc:1067] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Signalling peer a34c47dd29c14748a018d716ac6c39df to start an election
I20260812 19:07:31.450538 588 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.454308 30671 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34276
I20260812 19:07:31.454953 30671 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:31.455334 30671 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.455473 528 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:31.455936 528 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:31.456722 528 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.460790 466 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:31.461295 466 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:31.462270 466 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.470798 29576 raft_consensus.cc:1261] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:31.481356 525 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.482116 591 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:31.475929 596 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260812 19:07:31.500237 29574 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:31.504268 525 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:31.506089 29177 catalog_manager.cc:5495] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:31.507951 29177 catalog_manager.cc:5495] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 2 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:31.508878 466 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:31.509581 29177 catalog_manager.cc:5495] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 3 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:31.497805 512 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260812 19:07:31.515339 512 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:31.512452 528 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:31.552577 30671 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.579376 30686 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:31.580345 30671 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:31.581449 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:31.582132 29579 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.591337 580 consensus_queue.cc:1046] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:31.592379 588 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:31.592726 588 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.593501 588 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.594343 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:31.594866 588 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:31.601486 29579 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:31.602556 30623 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:31.603511 446 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:31.603995 446 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:31.604815 446 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:31.620852 29326 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:31.622337 475 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:07:31.652486 29331 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:31.664062 29179 catalog_manager.cc:5495] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 4 to 5, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 5 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:31.899037 588 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:31.899489 588 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.901376 588 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:31.901925 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:31.902252 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:31.902653 29446 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:31.902807 29569 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:31.903713 29282 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:31.904424 588 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:31.904760 588 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:31.904991 588 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:31.910427 588 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.912145 588 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:31.913254 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:31.913796 29569 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:31.913719 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:31.914446 29446 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:31.919698 29569 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:31.920753 29281 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:31.921115 29446 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:31.921449 588 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:31.922607 588 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:31.923266 588 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:31.933027 29177 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:31.986644 588 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:31.987216 588 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:31.990790 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:31.990943 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:31.991370 588 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:31.992341 29446 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:31.993218 29282 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:31.994308 588 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:31.994593 588 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:31.994849 588 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.000478 588 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:32.002992 588 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.003638 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.004092 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.004981 29446 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.011420 29446 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.012430 29282 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:32.013386 588 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:32.013754 588 raft_consensus.cc:686] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:32.014518 588 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:32.031309 29177 catalog_manager.cc:5495] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 3, now has a pending config: VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65), VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66), VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67), NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:32.041226 546 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.041698 546 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.043326 546 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.043989 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.044231 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.044739 29446 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:32.045555 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:32.046144 546 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:32.046506 546 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.046761 546 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.053535 546 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.057525 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.058132 29446 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.062273 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:32.065629 29446 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:32.067832 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:32.069085 533 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:32.069561 533 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:32.070462 533 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.070763 546 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.083263 29176 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:32.092391 29569 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:07:32.093690 546 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:32.109263 29341 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:07:32.110720 446 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:32.130960 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:32.173334 29446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:32.173851 29446 raft_consensus.cc:595] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:32.176131 29446 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:32.176590 29446 raft_consensus.cc:595] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:32.176820 29446 raft_consensus.cc:603] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:07:32.177655 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260812 19:07:32.178839 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:32.179344 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:32.219825 580 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.220439 580 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.222488 580 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.223858 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.223901 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.224419 29446 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:32.224603 29569 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:32.225481 29282 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:32.226222 580 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:32.226516 580 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.226795 580 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.234759 580 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.237314 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.237996 29446 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.242568 29446 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:32.237313 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.243147 29569 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.243593 29282 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:32.244472 588 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:32.244616 580 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.246227 588 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:32.247148 588 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.248996 29569 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:32.269235 29176 catalog_manager.cc:5495] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:32.291126 512 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.291702 512 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.293792 512 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.295347 528 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:32.295879 528 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.296947 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.297591 29572 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:32.297842 528 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.298465 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.299064 30686 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:32.299994 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.300704 29459 raft_consensus.cc:2354] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:32.301882 29528 leader_election.cc:398] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:32.302238 29528 leader_election.cc:401] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:32.302809 528 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.305258 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.307586 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.308677 29572 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:07:32.310281 512 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:32.310141 588 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:32.310727 512 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.311124 512 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.309830 528 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:32.333020 580 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.333573 580 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.335638 580 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.336661 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.337325 29569 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.337344 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.338686 29459 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:32.339745 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.341066 580 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:32.341432 580 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.341737 580 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.350762 512 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.352653 580 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.354261 512 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.354902 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:32.355497 30686 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.360978 512 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.361760 512 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.362107 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.364197 29569 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.365757 512 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.369753 588 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.370312 588 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.372484 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.374220 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.374817 588 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.376804 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.377403 29572 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.378309 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.378909 29579 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:32.380013 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.381184 593 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:32.381544 593 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.381831 593 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.381891 536 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.382524 536 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.386106 536 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.385840 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.393242 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.397518 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.398223 29459 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:32.400350 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:32.403848 29569 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:32.401206 30686 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:32.405426 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.405624 528 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:32.405215 621 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:32.409185 528 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.409713 528 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.409726 621 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.410355 621 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.412112 580 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.412835 475 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:32.430704 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.431509 29579 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.438447 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.439215 29459 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.440445 29579 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:32.440894 29446 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:07:32.441867 634 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.442699 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.445058 627 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:32.445515 627 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:32.446336 627 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.443578 634 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.450749 29459 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:32.455526 630 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:32.451342 580 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:32.456137 630 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.464449 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.453231 29579 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:32.465230 29446 raft_consensus.cc:2379] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:32.466341 588 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.480604 588 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.481477 634 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.482455 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.483889 593 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.484324 588 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.485113 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.485893 29446 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:32.487411 475 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:32.488929 475 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.490254 29451 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260812 19:07:32.505347 29281 consensus_peers.cc:487] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 -> Peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Couldn't send request to peer a34c47dd29c14748a018d716ac6c39df. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 005defc807dc47d881add64226efb7ce. This is attempt 1: this message will repeat every 5th retry.
I20260812 19:07:32.511680 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.512682 29569 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:32.487270 630 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.514302 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.492041 546 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260812 19:07:32.517462 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.518072 29569 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.518426 29281 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.519152 634 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:32.519526 634 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.519819 634 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.525484 634 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.531539 528 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.487703 621 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.532752 29569 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:32.533077 630 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.533986 621 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.533640 630 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.515723 536 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.535406 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.535358 536 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.535961 630 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.537047 533 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.537552 533 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.538767 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.539333 29451 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.539680 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.526643 635 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260812 19:07:32.541412 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.542150 30671 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.542490 593 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.542620 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.543457 637 raft_consensus.cc:979] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:32.543833 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.544359 638 raft_consensus.cc:1067] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer 6dd313c6592d4f33bab314a73f28b007 to start an election
I20260812 19:07:32.540899 29331 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.546020 512 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.546057 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.546604 512 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.548313 29579 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:32.548662 512 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.548988 536 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.550390 29335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
from {username='slave'} at 127.0.0.1:33260
I20260812 19:07:32.550930 29335 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:32.551251 29335 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.554250 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:32.554910 29326 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:07:32.559234 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.543958 637 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:32.561481 637 raft_consensus.cc:686] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:32.533749 528 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.562691 29451 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:32.562399 637 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.565280 533 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.542289 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.542088 29341 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:32.567003 30686 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:32.571825 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.566179 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.564733 29335 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.579430 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.578351 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.581405 29450 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:32.582055 645 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:32.582264 29335 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.582346 29528 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.576151 29331 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:32.583829 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:32.583151 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.576756 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.584821 29445 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:32.585181 29569 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.587064 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.587630 29572 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.590566 30629 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:32.588203 29176 catalog_manager.cc:5495] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 5, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 5 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:32.564669 446 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:32.590513 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.592867 30681 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:32.594421 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.594895 29450 raft_consensus.cc:3032] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:32.595155 29450 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:32.599264 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:32.599815 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.595971 29450 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.596441 29569 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:32.574740 621 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.604817 643 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.574450 580 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:32.580539 630 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:32.590744 627 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:32.579243 512 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.591054 546 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:32.593842 639 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:32.598229 650 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:32.628747 650 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.630088 533 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:32.630904 650 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.631405 650 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.631834 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.631839 650 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.615060 642 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:32.614213 630 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.632705 29335 raft_consensus.cc:2354] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
I20260812 19:07:32.632251 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.635756 29403 leader_election.cc:398] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
I20260812 19:07:32.636047 29403 leader_election.cc:401] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:32.602831 446 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.636638 593 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.638024 446 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.639402 30686 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.639703 546 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.616032 654 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:32.610136 643 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.648550 30686 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:32.640355 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.664834 29576 raft_consensus.cc:2354] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3.
I20260812 19:07:32.621805 638 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:32.623337 639 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.673995 30623 leader_election.cc:398] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3.
I20260812 19:07:32.674432 30623 leader_election.cc:401] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260812 19:07:32.675325 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 80589.1us
I20260812 19:07:32.643785 29326 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:32.624878 512 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.646205 654 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:32.625989 655 log.cc:924] Time spent T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260812 19:07:32.647315 634 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.633606 630 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:32.727896 29452 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:32.649569 643 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.604980 621 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.633229 642 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.650782 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.664219 580 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:32.651156 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:32.651811 546 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.653452 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.654486 29569 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 2: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:32.739993 29335 raft_consensus.cc:2354] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4.
I20260812 19:07:32.655433 29450 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.660566 29572 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.715057 645 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:32.745430 645 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.747144 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:32.717720 658 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3.
I20260812 19:07:32.722867 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:32.618289 627 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.750442 29450 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.673832 639 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:32.752189 512 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.673596 638 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:32.753726 638 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.756474 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:32.757179 29572 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:32.741297 29580 raft_consensus.cc:1226] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:32.743901 29574 raft_consensus.cc:2354] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4.
I20260812 19:07:32.744240 632 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 19:07:32.744628 29575 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:32.750531 627 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:32.729872 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:32.776077 29576 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.753341 635 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:32.765061 29452 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:32.770944 29281 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.723204 654 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:32.783033 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.786399 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.788218 675 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:32.798143 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 131562us
I20260812 19:07:32.739044 643 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.763118 580 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:32.764835 664 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.778537 661 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.844439 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:32.834210 637 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.834872 475 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.790735 675 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.848510 29576 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:32.866664 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:32.867583 30686 raft_consensus.cc:2379] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:32.837843 669 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.871055 29399 leader_election.cc:398] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4.
I20260812 19:07:32.880213 29399 leader_election.cc:401] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:32.871594 29176 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:32.869849 647 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.841293 643 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.902763 669 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.900637 512 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 4
W20260812 19:07:32.908597 655 log.cc:924] Time spent T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260812 19:07:32.909006 635 raft_consensus.cc:686] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:32.872026 637 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.895269 536 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.935904 29450 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:32.919796 593 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
I20260812 19:07:32.938760 687 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:32.907795 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 136920us
I20260812 19:07:32.914306 676 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.869022 666 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:32.878455 475 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.934273 669 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.945446 676 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.946031 29574 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:32.946171 536 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:32.956418 676 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.957064 536 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:32.910348 695 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:32.957854 695 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.960026 695 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:32.868703 664 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.962321 664 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:32.972941 695 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.973608 695 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.975850 695 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:32.980450 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 99157.7us
I20260812 19:07:32.864831 661 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.983107 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:32.983937 29459 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:32.935320 533 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.942816 634 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:32.951365 660 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:32.947738 666 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:32.997364 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.002053 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.002045 29281 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.003571 30686 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:32.952762 650 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.006464 29335 raft_consensus.cc:2379] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 2: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
W20260812 19:07:33.009442 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 253619us
I20260812 19:07:33.005110 29527 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.008729 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.013665 29575 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:32.930380 647 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.020489 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.022255 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.022902 29572 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:32.949139 687 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:32.998250 621 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.028301 29580 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:33.006482 666 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:33.038007 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.042361 29569 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:33.043875 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.044555 29335 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:32.995309 533 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.030551 687 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.056857 664 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:33.072877 664 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.009150 660 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.075129 664 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.062975 642 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:33.075541 670 log.cc:924] Time spent T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.145s user 0.000s sys 0.000s
I20260812 19:07:33.054062 678 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:33.077056 30623 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.037134 700 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260812 19:07:33.072808 680 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.159s user 0.000s sys 0.001s
I20260812 19:07:33.075845 696 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260812 19:07:33.075984 643 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.063902 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.067422 634 raft_consensus.cc:686] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:33.021107 635 consensus_queue.cc:237] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.085563 30629 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:33.094619 29445 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:33.094220 700 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:33.100692 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 94457.8us
I20260812 19:07:33.114391 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.116112 30686 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 2: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:33.117673 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.119484 29572 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:33.093807 647 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.094868 536 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:33.096045 675 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.128083 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.136739 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.137390 30681 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:33.138757 29527 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.113621 676 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:33.099622 678 raft_consensus.cc:686] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:33.118260 475 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.100700 30671 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
W20260812 19:07:33.141501 681 log.cc:924] Time spent T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.133s user 0.000s sys 0.001s
I20260812 19:07:33.118626 700 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:33.139163 661 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.155435 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.137091 713 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:33.170248 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 130679us
I20260812 19:07:33.122102 709 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:33.135617 637 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.141156 533 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.180160 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.142457 676 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.138271 536 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.175972 704 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:33.195580 29459 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:33.189247 29403 leader_election.cc:398] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4.
I20260812 19:07:33.189436 676 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:33.180028 709 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.179812 713 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:33.138135 711 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260812 19:07:33.202517 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.210968 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.214553 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.213410 29341 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:33.217923 627 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.219868 627 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.202350 728 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.220561 728 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.220908 728 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.222349 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.222685 29403 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:33.223138 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 102021us
I20260812 19:07:33.226186 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.226745 29459 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 6
I20260812 19:07:33.225075 29339 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:33.226861 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:33.231118 536 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.229357 673 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.200490 637 raft_consensus.cc:465] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Not starting pre-election -- already a leader
I20260812 19:07:33.238556 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.188149 630 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.242127 30686 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:33.175688 634 consensus_queue.cc:237] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.203008 709 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:33.245088 29527 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.205937 713 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:33.243731 29569 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:33.261554 29459 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:33.263312 29282 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:33.299227 734 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:33.245623 29326 raft_consensus.cc:2354] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:33.243294 475 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:33.261992 673 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.227069 588 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260812 19:07:33.243472 660 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.301298 29337 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:33.201299 704 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:33.301169 638 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.547s user 0.011s sys 0.000s
I20260812 19:07:33.221657 678 consensus_queue.cc:237] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.320978 29403 leader_election.cc:398] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:33.320729 638 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.321290 29403 leader_election.cc:401] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:33.321866 643 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:33.322903 638 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.323505 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:33.301460 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:33.325842 639 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.568s user 0.010s sys 0.009s
I20260812 19:07:33.310225 475 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:33.312454 734 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.333370 734 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:33.332866 639 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.336740 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:33.314343 673 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.339424 639 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:33.341738 737 ts_tablet_manager.cc:927] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
I20260812 19:07:33.344296 714 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:33.344836 714 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:33.332924 475 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:33.340148 30671 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:33.345690 714 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.382936 29527 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.406920 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.416884 29326 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260812 19:07:33.462625 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 79784.7us
I20260812 19:07:33.465773 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.419433 533 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.467172 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.471567 29567 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:33.471969 29575 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 1.
I20260812 19:07:33.469285 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.470824 30625 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:33.472906 29578 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:33.473564 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.473977 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.476476 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.478427 684 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:33.478868 684 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.479198 684 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:33.480795 700 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.482563 29452 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:33.483052 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.487645 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.451246 699 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260812 19:07:33.494742 704 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.496413 737 tablet_copy_client.cc:316] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: tablet copy: Beginning tablet copy session from remote peer at address 127.26.205.65:44295
I20260812 19:07:33.483667 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.474148 533 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.502239 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.502858 29457 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:33.504959 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.487056 643 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.500473 533 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.508744 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.498837 546 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:33.510686 29453 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:33.510986 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.512817 29457 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:33.512526 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.518922 29456 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:33.523846 29355 tablet_copy_service.cc:140] P 6dd313c6592d4f33bab314a73f28b007: Received BeginTabletCopySession request for tablet 005defc807dc47d881add64226efb7ce from peer a34c47dd29c14748a018d716ac6c39df ({username='slave'} at 127.0.0.1:41568)
I20260812 19:07:33.524489 29355 tablet_copy_service.cc:161] P 6dd313c6592d4f33bab314a73f28b007: Beginning new tablet copy session on tablet 005defc807dc47d881add64226efb7ce from peer a34c47dd29c14748a018d716ac6c39df at {username='slave'} at 127.0.0.1:41568: session id = a34c47dd29c14748a018d716ac6c39df-005defc807dc47d881add64226efb7ce
W20260812 19:07:33.525002 580 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.681s user 0.009s sys 0.006s
I20260812 19:07:33.509258 643 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.508618 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.531940 580 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.543090 29334 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:33.537410 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.510105 630 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.512691 546 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:33.518756 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.549311 709 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.555938 30679 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:33.549433 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.551638 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.554915 29334 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:33.557372 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.510746 713 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:33.557035 546 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:07:33.547299 512 consensus_meta.cc:287] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.615s user 0.024s sys 0.001s
I20260812 19:07:33.537863 643 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.558362 29453 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
W20260812 19:07:33.561110 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 417495us
I20260812 19:07:33.560207 29338 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:33.563486 29459 raft_consensus.cc:2354] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:33.542626 666 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.572734 29446 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:33.573237 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.578649 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.592754 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2172) took 1039 ms. Trace:
I20260812 19:07:33.586987 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 1984) took 1006 ms. Trace:
I20260812 19:07:33.593395 29453 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:33.577684 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:33.578807 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:33.595467 29578 raft_consensus.cc:2354] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:33.560060 713 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.596906 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.599828 29457 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:33.583308 30675 raft_consensus.cc:2354] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:33.593463 29579 rpcz_store.cc:276] 0812 19:07:32.580209 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:32.580342 (+ 133us) service_pool.cc:227] Handling call
0812 19:07:32.583872 (+ 3530us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:32.736510 (+152638us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:32.741661 (+ 5151us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:32.742797 (+ 1136us) log.cc:842] Serialized 532 byte log entry
0812 19:07:32.786087 (+ 43290us) raft_consensus.cc:1631] Marking committed up to 1
0812 19:07:32.786127 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:32.786205 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:33.584946 (+798741us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b24001e5ca8. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66d4acf4 00007f9c66d48dfb 00007f9c66dd7559 00007f9c66dd71e5 00007f9c66dd6740 00007f9c69abd578 00007f9c69abdcd8 00007f9c69a956e8 00007f9c69a90aea 00007f9c69a8ff36
0812 19:07:33.586703 (+ 1757us) raft_consensus.cc:1671] finished
0812 19:07:33.586722 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:33.586929 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5998,"raft.queue_time_us":42464,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":3463552,"thread_start_us":188969,"threads_started":2,"wal-append.queue_time_us":100735}
I20260812 19:07:33.531246 734 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.595953 29337 rpcz_store.cc:276] 0812 19:07:32.553777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:32.571096 (+ 17319us) service_pool.cc:227] Handling call
0812 19:07:33.592598 (+1021502us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400377dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:33.592697 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1537889536}
I20260812 19:07:33.567819 700 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.571244 29282 leader_election.cc:398] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:33.598479 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:33.571125 512 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4.
I20260812 19:07:33.603447 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:33.568131 580 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.588474 29355 tablet_copy_source_session.cc:205] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 19:07:33.610682 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.613603 29459 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:33.614322 29282 leader_election.cc:401] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260812 19:07:33.573243 666 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.596309 684 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.615150 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:33.618892 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:33.610723 654 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.773s user 0.011s sys 0.034s
I20260812 19:07:33.718919 29578 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:33.719738 29569 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:33.720835 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.575701 673 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:33.722716 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 152223us
I20260812 19:07:33.626988 29281 leader_election.cc:398] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:33.749629 29281 leader_election.cc:401] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260812 19:07:33.728160 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:33.750245 29459 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:33.753720 29453 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:33.756706 29572 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:33.758911 29528 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:33.627381 30674 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:33.766033 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:33.766754 29579 raft_consensus.cc:2354] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:33.766767 30625 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:33.765594 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.771206 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.768956 29459 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:33.749644 580 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:33.775050 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:33.777042 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:33.605963 533 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.780370 30681 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:33.223655 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:33.769915 661 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:33.802924 29528 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:33.805110 30686 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:33.750429 654 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.791281 669 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.819001 29459 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260812 19:07:33.600899 728 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.813546 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:33.825567 29331 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:33.803563 630 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:33.781688 666 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260812 19:07:33.812778 661 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:33.859321 30679 raft_consensus.cc:2354] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:33.860819 654 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:33.867307 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:33.868046 728 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.870060 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:33.861697 661 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:33.833118 678 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260812 19:07:33.875793 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:33.876410 30681 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.822562 669 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:33.850885 699 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:33.885488 723 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260812 19:07:33.890051 676 consensus_meta.cc:287] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.684s user 0.009s sys 0.002s
I20260812 19:07:33.890607 676 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.892819 29575 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:33.893812 475 consensus_meta.cc:287] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.538s user 0.007s sys 0.039s
I20260812 19:07:33.831039 664 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260812 19:07:33.893918 759 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:33.902350 708 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:33.898548 723 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.894308 686 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:33.884712 669 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.926306 30681 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:33.887836 709 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:33.927696 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:33.930413 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:33.931197 757 log.cc:924] Time spent T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.192s user 0.002s sys 0.000s
I20260812 19:07:33.927608 720 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:33.935240 720 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.935781 720 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:33.938004 29331 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:33.919273 676 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.920141 30671 raft_consensus.cc:2379] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:33.921243 29459 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:33.943387 29453 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:33.921532 737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 005defc807dc47d881add64226efb7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 19:07:33.913062 723 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.924386 686 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:33.899437 699 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:33.911525 475 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.946444 767 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:33.947175 30629 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:33.951393 740 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.315s user 0.001s sys 0.001s
W20260812 19:07:33.952306 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 174347us
I20260812 19:07:33.973663 29399 leader_election.cc:398] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:33.974043 29399 leader_election.cc:401] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260812 19:07:33.963855 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:33.964195 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 161374us
I20260812 19:07:33.937273 734 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.979841 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:33.942778 533 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:33.982986 29451 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:33.982052 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:33.955528 686 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:33.943359 759 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:33.964861 684 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:33.985984 638 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:33.986393 734 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:33.987367 767 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:07:34.013496 750 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.238s user 0.000s sys 0.002s
I20260812 19:07:34.023415 702 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:34.024061 702 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:34.009982 687 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:34.026053 666 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:34.030603 666 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:34.031054 666 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:34.024993 702 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:34.032338 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 232804us
I20260812 19:07:34.003242 621 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260812 19:07:34.025064 687 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.016108 734 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.022099 699 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.022708 669 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.027643 546 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.027065 763 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.104s
I20260812 19:07:34.033612 643 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.089249 767 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.102476 30681 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:34.111994 687 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:34.090293 475 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.103240 29403 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:34.122013 684 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:34.123553 30684 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:34.089744 638 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:34.129385 29452 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:34.122831 684 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.130287 475 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:34.131006 475 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:34.132124 546 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:34.132565 475 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.152791 30681 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:34.122283 723 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.100178 705 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:34.184808 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:34.185660 30686 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:34.187057 29527 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:34.206558 29453 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:34.135003 638 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.214911 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:34.116058 720 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.207948 528 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:34.190760 684 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.211491 533 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:34.247331 528 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:34.271860 528 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:34.261502 533 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.244719 687 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:34.278420 687 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.279569 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:34.277588 533 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:34.306502 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 386) took 1471 ms. Trace:
I20260812 19:07:34.308144 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:34.308198 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:34.309670 30684 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:34.312829 29527 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:34.306989 29176 rpcz_store.cc:276] 0812 19:07:32.834689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:32.835050 (+ 361us) service_pool.cc:227] Handling call
0812 19:07:34.306420 (+1471370us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:32.908455 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:32.939431 (+ 30976us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:32.939468 (+ 37us) write_op.cc:435] Acquired schema lock
0812 19:07:32.939485 (+ 17us) tablet.cc:660] Decoding operations
0812 19:07:32.949317 (+ 9832us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:32.982139 (+ 32822us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:32.984606 (+ 2467us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:33.006096 (+ 21490us) tablet.cc:699] Row locks acquired
0812 19:07:33.006117 (+ 21us) write_op.cc:260] PREPARE: finished
0812 19:07:33.020215 (+ 14098us) write_op.cc:270] Start()
0812 19:07:33.033249 (+ 13034us) write_op.cc:276] Timestamp: P: 1786561653017629 usec, L: 0
0812 19:07:33.033292 (+ 43us) op_driver.cc:331] REPLICATION: starting
0812 19:07:33.091181 (+ 57889us) log.cc:842] Serialized 536 byte log entry
0812 19:07:33.347402 (+256221us) op_driver.cc:444] REPLICATION: finished
0812 19:07:33.514386 (+166984us) write_op.cc:301] APPLY: starting
0812 19:07:33.514477 (+ 91us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:33.514585 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:33.514729 (+ 144us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:33.514938 (+ 209us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:33.515151 (+ 213us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:33.515197 (+ 46us) write_op.cc:312] APPLY: finished
0812 19:07:33.515410 (+ 213us) log.cc:842] Serialized 23 byte log entry
0812 19:07:33.516706 (+ 1296us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:33.516791 (+ 85us) write_op.cc:454] Released schema lock
0812 19:07:33.516833 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:33.517092 (+ 259us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:33.517132 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":268017,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":158958,"apply.run_cpu_time_us":1880,"apply.run_wall_time_us":2992,"num_ops":1,"prepare.queue_time_us":10678,"prepare.run_cpu_time_us":3557,"prepare.run_wall_time_us":191394,"replication_time_us":295544,"thread_start_us":124256,"threads_started":1}]]}
I20260812 19:07:34.259030 759 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:34.269346 734 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:34.283030 687 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:34.309008 30674 consensus_meta.cc:287] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.522s user 0.014s sys 0.000s
I20260812 19:07:34.288473 669 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:34.339087 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:34.346321 30684 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:34.311725 643 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.344993 723 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.318532 684 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:34.335701 673 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.368054 708 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:34.368999 708 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:34.373580 720 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:34.341035 630 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:34.352852 684 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:34.348235 30674 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:34.389082 684 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:34.399713 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.631s user 0.011s sys 0.014s
W20260812 19:07:34.400121 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.631s user 0.011s sys 0.014s
W20260812 19:07:34.401895 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 633088us
I20260812 19:07:34.395458 638 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:34.398984 720 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:34.416136 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.663s user 0.008s sys 0.037s
W20260812 19:07:34.434331 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.681s user 0.008s sys 0.037s
W20260812 19:07:34.435094 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 808201us
W20260812 19:07:34.444898 661 consensus_meta.cc:287] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.573s user 0.009s sys 0.006s
W20260812 19:07:34.459355 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 422528us
W20260812 19:07:34.466780 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.663s user 0.006s sys 0.015s
W20260812 19:07:34.467128 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.663s user 0.006s sys 0.015s
W20260812 19:07:34.467387 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 673130us
I20260812 19:07:34.467972 29451 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260812 19:07:34.468561 30623 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:34.468753 29458 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.015s sys 0.026s
W20260812 19:07:34.469136 29458 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s user 0.015s sys 0.026s
I20260812 19:07:34.436043 720 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:34.495229 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 515429us
I20260812 19:07:34.497027 785 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:34.497893 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:34.498669 723 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:34.499069 723 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:34.500474 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:34.456372 726 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:34.503682 29281 leader_election.cc:398] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:34.507225 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:34.507911 30684 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:34.508297 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:34.512948 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.513126 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.517490 29580 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:34.520077 29451 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:34.520737 29451 raft_consensus.cc:595] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:34.521880 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.522703 29451 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:34.523702 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.525529 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.525764 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:34.499395 723 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:34.528106 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:34.515976 726 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.530213 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:34.531111 776 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:34.532007 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:34.494452 533 raft_consensus.cc:2741] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260812 19:07:34.591604 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:34.541215 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:34.542806 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.593319 29580 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260812 19:07:34.545192 666 consensus_meta.cc:287] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.514s user 0.004s sys 0.008s
I20260812 19:07:34.546718 30684 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:34.597153 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:34.456545 661 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.525134 643 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:34.585765 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.579661 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.534439 708 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.596033 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:34.597477 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:34.597990 29331 raft_consensus.cc:2354] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:34.595561 29580 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260812 19:07:34.586149 686 consensus_meta.cc:287] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.591s user 0.015s sys 0.000s
I20260812 19:07:34.525681 660 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:34.523957 767 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:34.661620 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 68411.8us
I20260812 19:07:34.663987 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:34.620292 29339 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:34.623792 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:34.624908 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.628509 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:34.629987 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:34.669456 29580 raft_consensus.cc:2354] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:07:34.580013 664 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:34.633920 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.682811 29575 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:34.707116 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.708029 29456 raft_consensus.cc:2379] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:34.604106 666 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.722308 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.723791 730 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:34.724499 708 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:34.720743 30674 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:34.606038 536 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:34.725597 29572 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:07:34.733480 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 141961us
I20260812 19:07:34.685623 664 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.749442 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:34.752249 29459 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:34.752279 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:34.744617 782 log.cc:924] Time spent T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.190s user 0.000s sys 0.001s
I20260812 19:07:34.703187 29326 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:34.746049 637 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:34.743566 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:34.686722 759 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:34.756817 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:34.775442 29445 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:34.733667 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:34.711385 687 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:34.625577 643 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.762822 29577 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:34.720649 661 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.681074 660 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.786993 29453 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:34.604650 776 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.685235 767 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:34.852003 726 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:34.631765 686 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.876161 29575 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:34.773051 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 740) took 1274 ms. Trace:
I20260812 19:07:34.773232 730 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:34.940142 29451 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:34.961138 29339 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:34.787834 759 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:34.894295 30675 rpcz_store.cc:276] 0812 19:07:33.488559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.490628 (+ 2069us) service_pool.cc:227] Handling call
0812 19:07:34.748554 (+1257926us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b5400480dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:34.758051 (+ 9497us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":100866944}
I20260812 19:07:34.949856 676 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:34.946905 720 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:34.996191 30670 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:35.002024 637 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.007854 647 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.008533 647 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.004442 788 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.161s user 0.000s sys 0.001s
I20260812 19:07:34.809732 687 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:35.004699 792 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.104s user 0.000s sys 0.001s
W20260812 19:07:35.005431 684 consensus_meta.cc:287] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.596s user 0.008s sys 0.004s
I20260812 19:07:35.010296 730 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.035176 475 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.036042 475 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.040266 759 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.040856 475 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.041235 661 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:35.041908 661 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:35.045742 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1811) took 1575 ms. Trace:
I20260812 19:07:35.046162 29458 rpcz_store.cc:276] 0812 19:07:33.460929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.461094 (+ 165us) service_pool.cc:227] Handling call
0812 19:07:33.936370 (+475276us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:33.936579 (+ 209us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:33.936930 (+ 351us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:33.936970 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:33.937406 (+ 436us) log.cc:842] Serialized 26 byte log entry
0812 19:07:34.469686 (+532280us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:34.469725 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:34.469790 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:35.002651 (+532861us) raft_consensus.cc:1671] finished
0812 19:07:35.002672 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:35.036239 (+ 33567us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6284,"raft.queue_time_us":5224,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":9433,"spinlock_wait_cycles":2561920,"thread_start_us":503863,"threads_started":1,"wal-append.queue_time_us":591565}
I20260812 19:07:34.926965 29445 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:35.045295 588 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:35.049098 588 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.049698 660 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.034673 684 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.051330 29569 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260812 19:07:35.052632 664 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.053510 664 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:35.054483 686 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.054127 664 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.055816 765 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.057479 720 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:35.049815 528 consensus_meta.cc:287] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.777s user 0.012s sys 0.008s
I20260812 19:07:35.051823 687 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.051128 666 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.050601 660 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:35.056052 588 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.038422 767 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:35.103963 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.027s sys 0.062s
W20260812 19:07:35.086030 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 403480us
W20260812 19:07:35.104300 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.506s user 0.027s sys 0.062s
W20260812 19:07:35.090844 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 495611us
I20260812 19:07:35.096781 684 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.097445 528 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.077656 475 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.045164 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:35.007321 637 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.052788 643 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.084580 785 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:07:35.104698 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 568455us
I20260812 19:07:35.110989 776 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.112542 666 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:35.014478 676 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.124404 705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:41558 (request call id 13) took 4766 ms. Trace:
I20260812 19:07:35.131311 637 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:35.129156 723 consensus_meta.cc:287] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.584s user 0.011s sys 0.011s
I20260812 19:07:35.157358 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:35.158361 29450 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:35.099922 647 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.137364 29578 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:35.135118 688 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260812 19:07:35.137661 666 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.139371 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:35.057415 765 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.148924 676 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.106612 660 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:35.156082 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:35.145221 705 rpcz_store.cc:276] 0812 19:07:30.348562 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:30.370649 (+ 22087us) service_pool.cc:227] Handling call
0812 19:07:30.441595 (+ 70946us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b580021b3b0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d89f63 00007f9c69dedaba 00007f9c69e09a7a 00007f9c69e0aada 00007f9c69e09c8b 00007f9c69e15084 00007f9c69e14743 00007f9c6ed50f39 00007f9c68448704 00007f9c68448692
0812 19:07:30.462782 (+ 21187us) log.cc:842] Serialized 506 byte log entry
0812 19:07:35.113617 (+4650835us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1782,"raft.run_cpu_time_us":1290,"raft.run_wall_time_us":1287,"spinlock_wait_cycles":149308288,"thread_start_us":30954,"threads_started":2,"wal-append.queue_time_us":22477,"wal-append.run_cpu_time_us":1670,"wal-append.run_wall_time_us":1004167}
I20260812 19:07:35.095590 720 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.164610 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 405958us
W20260812 19:07:35.204434 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 65170.3us
W20260812 19:07:35.173869 29446 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s user 0.004s sys 0.027s
I20260812 19:07:35.155872 29403 leader_election.cc:398] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:35.128332 29574 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:35.160578 29445 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
W20260812 19:07:35.205772 29446 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s user 0.004s sys 0.027s
I20260812 19:07:35.205978 29403 leader_election.cc:401] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:07:35.206736 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.162124 723 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.210031 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 54257.2us
I20260812 19:07:35.211053 29403 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:35.213217 29335 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:35.214368 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.215063 29339 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:35.215565 702 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:35.215889 643 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:35.216296 546 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:35.217244 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.217067 546 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.234436 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.188969 661 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.234189 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:35.245432 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:35.247797 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:35.233731 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.235916 29457 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:35.136770 785 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.238770 29337 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:07:35.239563 723 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.216897 643 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:35.240015 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.240168 29177 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53840 (request call id 289) took 2035 ms. Trace:
I20260812 19:07:35.263365 29177 rpcz_store.cc:276] 0812 19:07:33.204352 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.204839 (+ 487us) service_pool.cc:227] Handling call
0812 19:07:35.240104 (+2035265us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":555658,"reported_tablets":1}
I20260812 19:07:35.200357 627 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.240913 702 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.218425 647 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:35.210678 714 raft_consensus.cc:2741] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260812 19:07:35.211014 759 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:35.261257 765 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.246148 29174 catalog_manager.cc:5495] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:35.239591 546 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.290722 29332 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:07:35.297617 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.017s sys 0.035s
I20260812 19:07:35.118670 475 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.540923 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.859s user 0.017s sys 0.051s
I20260812 19:07:35.523020 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2298) took 1004 ms. Trace:
W20260812 19:07:35.543452 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 914955us
W20260812 19:07:35.540566 29456 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.771s user 0.013s sys 0.073s
I20260812 19:07:35.285136 650 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:35.523021 29282 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:35.522483 29403 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:35.544430 29456 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s user 0.014s sys 0.074s
W20260812 19:07:35.376932 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.003s sys 0.011s
W20260812 19:07:35.555555 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.783s user 0.007s sys 0.011s
I20260812 19:07:35.543669 29446 rpcz_store.cc:276] 0812 19:07:34.518783 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.518917 (+ 134us) service_pool.cc:227] Handling call
0812 19:07:34.526236 (+ 7319us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:34.526654 (+ 418us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.526682 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:34.527111 (+ 429us) log.cc:842] Serialized 26 byte log entry
0812 19:07:35.208838 (+681727us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:35.208874 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:35.208933 (+ 59us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:35.522813 (+313880us) raft_consensus.cc:1671] finished
0812 19:07:35.522833 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:35.522961 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":36008,"raft.queue_time_us":10821,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":39499,"spinlock_wait_cycles":1506304,"thread_start_us":604857,"threads_started":1,"wal-append.queue_time_us":931694}
I20260812 19:07:35.351384 528 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.284962 627 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.583948 627 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:35.590752 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 67822.2us
W20260812 19:07:35.590675 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 261478us
I20260812 19:07:35.584780 29342 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
W20260812 19:07:35.551373 704 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.691s user 0.000s sys 0.016s
I20260812 19:07:35.300007 664 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.599000 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2301) took 1073 ms. Trace:
I20260812 19:07:35.619066 29451 rpcz_store.cc:276] 0812 19:07:34.519780 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.527868 (+ 8088us) service_pool.cc:227] Handling call
0812 19:07:35.592813 (+1064945us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54003f3c54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:35.592919 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":928567552}
I20260812 19:07:35.599411 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2297) took 1077 ms. Trace:
I20260812 19:07:35.598819 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2296) took 1074 ms. Trace:
I20260812 19:07:35.561014 650 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:35.595522 714 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:35.602186 675 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.603336 635 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.530236 637 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260812 19:07:35.615429 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:35.620837 29459 rpcz_store.cc:276] 0812 19:07:34.518053 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.518177 (+ 124us) service_pool.cc:227] Handling call
0812 19:07:35.592558 (+1074381us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54000fa5d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:35.592687 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":516767232}
I20260812 19:07:35.620144 29457 rpcz_store.cc:276] 0812 19:07:34.518488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.522145 (+ 3657us) service_pool.cc:227] Handling call
0812 19:07:35.595956 (+1073811us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54003f4b54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:35.596100 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1449298688}
I20260812 19:07:35.310927 759 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.598320 704 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.809s user 0.000s sys 0.017s
I20260812 19:07:35.597905 588 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260812 19:07:35.631842 666 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.650262 29176 catalog_manager.cc:5495] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:35.600314 676 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:35.616344 647 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:35.654953 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:35.664433 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 576) took 1126 ms. Trace:
I20260812 19:07:35.648952 675 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.667346 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:35.657055 29326 consensus_meta.cc:287] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.868s user 0.004s sys 0.008s
I20260812 19:07:35.671295 29326 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:35.631670 688 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260812 19:07:35.674335 688 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:35.627409 714 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.657501 650 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.658509 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:35.545076 643 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:35.623754 660 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.664845 29334 rpcz_store.cc:276] 0812 19:07:34.535930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.536102 (+ 172us) service_pool.cc:227] Handling call
0812 19:07:34.543943 (+ 7841us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:34.940334 (+396391us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.947435 (+ 7101us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:34.947485 (+ 50us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:34.947528 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:35.661684 (+714156us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540039bcd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:35.661731 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:35.661861 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":911187200}
I20260812 19:07:35.631513 720 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.672179 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2186) took 1078 ms. Trace:
I20260812 19:07:35.649148 635 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.797863 737 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.638s user 0.012s sys 0.000s
I20260812 19:07:35.664674 647 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.686976 714 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:35.677829 678 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:35.807492 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.036s user 0.003s sys 0.040s
W20260812 19:07:35.810844 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.039s user 0.004s sys 0.040s
W20260812 19:07:35.811245 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.0437e+06us
W20260812 19:07:35.807898 29528 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.867s user 0.000s sys 0.044s
W20260812 19:07:35.812410 29528 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.872s user 0.000s sys 0.045s
W20260812 19:07:35.801721 696 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s user 0.009s sys 0.018s
W20260812 19:07:35.834122 696 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.699s user 0.009s sys 0.018s
I20260812 19:07:35.799989 704 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:35.809226 647 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:35.669441 676 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.892918 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.934s user 0.011s sys 0.017s
W20260812 19:07:35.897426 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.938s user 0.011s sys 0.017s
W20260812 19:07:35.865553 705 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s user 0.012s sys 0.010s
I20260812 19:07:35.817269 702 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:35.909053 705 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.648s user 0.012s sys 0.010s
I20260812 19:07:35.674979 688 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.866470 546 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:35.897656 704 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.934684 30671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 714) took 2316 ms (client timeout 1800 ms). Trace:
I20260812 19:07:35.898939 660 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:35.938609 675 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.939517 688 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.866955 797 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260812 19:07:35.936139 759 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:35.945024 660 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.963189 660 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.967607 704 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.965613 676 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:35.933200 785 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:35.982788 737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:34276 (request call id 668) took 2860 ms. Trace:
W20260812 19:07:36.027187 29400 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s user 0.014s sys 0.011s
W20260812 19:07:36.027987 29400 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.521s user 0.014s sys 0.011s
I20260812 19:07:36.032671 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1825) took 1434 ms. Trace:
I20260812 19:07:36.032775 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.034443 29451 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:36.033406 29456 rpcz_store.cc:276] 0812 19:07:34.598333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.767041 (+168708us) service_pool.cc:227] Handling call
0812 19:07:34.767599 (+ 558us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:34.768220 (+ 621us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.768249 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:34.768742 (+ 493us) log.cc:842] Serialized 26 byte log entry
0812 19:07:35.554220 (+785478us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:35.554257 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:35.554322 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:36.031986 (+477664us) raft_consensus.cc:1671] finished
0812 19:07:36.032055 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.032397 (+ 342us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":88,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119,"thread_start_us":785130,"threads_started":1,"wal-append.queue_time_us":1130771}
I20260812 19:07:35.939694 475 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.055241 688 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.056078 785 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.056329 688 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:36.056934 785 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:35.932732 546 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:35.983733 737 rpcz_store.cc:276] 0812 19:07:33.134219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.136451 (+ 2232us) service_pool.cc:227] Handling call
0812 19:07:33.342103 (+205652us) ts_tablet_manager.cc:928] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
0812 19:07:33.855397 (+513294us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:07:35.134928 (+1279531us) tablet_metadata.cc:641] Metadata flushed
0812 19:07:35.135652 (+ 724us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:07:35.938219 (+802567us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:07:35.980614 (+ 42395us) tablet_replica.cc:205] Creating consensus instance
0812 19:07:35.981282 (+ 668us) ts_tablet_manager.cc:983] Replica registered.
0812 19:07:35.982601 (+ 1319us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":249,"tablet-copy.queue_time_us":74960,"thread_start_us":86874,"threads_started":2}
I20260812 19:07:35.828344 678 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:35.935266 30671 rpcz_store.cc:269] 0812 19:07:33.618388 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.618557 (+ 169us) service_pool.cc:227] Handling call
0812 19:07:35.934272 (+2315715us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54005c1a54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:35.934482 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":406628224}
I20260812 19:07:35.794775 29326 rpcz_store.cc:276] 0812 19:07:34.593887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.596532 (+ 2645us) service_pool.cc:227] Handling call
0812 19:07:35.672024 (+1075492us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.074630 714 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:36.129071 29450 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.827s user 0.005s sys 0.021s
W20260812 19:07:36.136840 29450 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.932s user 0.005s sys 0.023s
I20260812 19:07:36.136914 737 tablet_copy_client.cc:791] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 0 data blocks...
I20260812 19:07:36.129990 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.142907 714 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.151185 737 tablet_copy_client.cc:655] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: tablet copy: Starting download of 1 WAL segments...
I20260812 19:07:36.174909 711 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.175356 546 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.175864 711 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.176507 635 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.177286 588 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:36.178145 588 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.180775 588 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.182307 688 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.184340 678 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:36.192020 30670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 739) took 2703 ms (client timeout 1944 ms). Trace:
W20260812 19:07:36.170670 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 55654.5us
I20260812 19:07:36.192046 821 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:36.217345 29445 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s user 0.010s sys 0.007s
W20260812 19:07:36.220077 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.635s user 0.004s sys 0.010s
W20260812 19:07:36.226069 29445 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s user 0.010s sys 0.007s
W20260812 19:07:36.194857 30670 rpcz_store.cc:269] 0812 19:07:33.488146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.490667 (+ 2521us) service_pool.cc:227] Handling call
0812 19:07:33.869835 (+379168us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66ebabce 000000000034aea9 00007f9c6ecdaa30 00007f9c6eda07b4 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:36.191475 (+2321640us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540072d8d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.191746 (+ 271us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2524377984}
W20260812 19:07:36.228989 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.651s user 0.005s sys 0.010s
I20260812 19:07:36.226600 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.227025 785 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:36.236645 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.214s user 0.006s sys 0.044s
W20260812 19:07:36.243093 29331 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.922s user 0.024s sys 0.029s
W20260812 19:07:36.244176 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.285s user 0.007s sys 0.044s
W20260812 19:07:36.239990 710 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.932s user 0.024s sys 0.065s
I20260812 19:07:36.232821 711 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.256844 825 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.261350 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.261044 825 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.262478 29451 raft_consensus.cc:2354] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:36.262820 512 raft_consensus.cc:979] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
W20260812 19:07:36.263264 29331 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.973s user 0.025s sys 0.030s
I20260812 19:07:36.264057 825 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.264285 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.264693 825 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:36.265202 29174 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.955s user 0.008s sys 0.016s
W20260812 19:07:36.266027 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.586s user 0.003s sys 0.016s
W20260812 19:07:36.266396 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.641s user 0.003s sys 0.016s
I20260812 19:07:36.293738 593 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.266827 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.265496 825 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:36.258111 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.596s user 0.010s sys 0.014s
W20260812 19:07:36.310365 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 92737.9us
I20260812 19:07:36.310365 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.258185 29456 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:36.263441 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:36.302675 29174 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.010s user 0.008s sys 0.023s
I20260812 19:07:36.311087 512 raft_consensus.cc:1067] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:36.305097 593 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.340935 593 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:36.332772 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.683s user 0.012s sys 0.021s
I20260812 19:07:36.346169 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.347517 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.244946 821 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.348884 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.351161 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.352084 29341 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:36.352270 30623 leader_election.cc:398] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:07:36.352744 30623 leader_election.cc:401] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260812 19:07:36.258317 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.003s sys 0.011s
I20260812 19:07:36.313573 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1826) took 1158 ms. Trace:
I20260812 19:07:36.356340 29445 rpcz_store.cc:276] 0812 19:07:35.154825 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.159913 (+ 5088us) service_pool.cc:227] Handling call
0812 19:07:35.160209 (+ 296us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:35.160457 (+ 248us) raft_consensus.cc:1520] Early marking committed up to index 3
0812 19:07:35.384811 (+224354us) log.cc:842] Serialized 14 byte log entry
0812 19:07:36.312806 (+927995us) log.cc:842] Serialized 14 byte log entry
0812 19:07:36.313145 (+ 339us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.313215 (+ 70us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:36.313290 (+ 75us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.313369 (+ 79us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.313512 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":10526,"prepare.queue_time_us":198,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":73,"raft.queue_time_us":28473,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":57340,"spinlock_wait_cycles":3562752,"thread_start_us":841397,"threads_started":1}
W20260812 19:07:36.333036 29403 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.000s sys 0.032s
W20260812 19:07:36.361044 29403 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.783s user 0.000s sys 0.033s
I20260812 19:07:36.362203 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1830) took 1186 ms. Trace:
I20260812 19:07:36.336653 686 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:36.368468 29572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2063) took 1845 ms (client timeout 2019 ms). Trace:
W20260812 19:07:36.258420 710 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.996s user 0.024s sys 0.066s
I20260812 19:07:36.337006 29326 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:07:36.338654 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:36.362407 29283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.635s user 0.001s sys 0.010s
I20260812 19:07:36.293118 730 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.363154 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2057) took 2602 ms. Trace:
W20260812 19:07:36.371565 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.763s user 0.008s sys 0.015s
I20260812 19:07:36.363857 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:36.258994 29402 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.619s user 0.004s sys 0.006s
I20260812 19:07:36.365010 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2059) took 1864 ms. Trace:
I20260812 19:07:36.362787 593 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.311689 728 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.359244 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.342463 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.368176 686 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.350327 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.350611 821 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:36.377153 29402 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.789s user 0.008s sys 0.012s
W20260812 19:07:36.359165 789 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 1.826s user 0.021s sys 0.032s
W20260812 19:07:36.368889 29572 rpcz_store.cc:269] 0812 19:07:34.523287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.523432 (+ 145us) service_pool.cc:227] Handling call
0812 19:07:36.367995 (+1844563us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002008d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.368258 (+ 263us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":374183168}
I20260812 19:07:36.372787 29569 rpcz_store.cc:276] 0812 19:07:34.500407 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.500819 (+ 412us) service_pool.cc:227] Handling call
0812 19:07:34.502743 (+ 1924us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:35.051755 (+549012us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.364433 (+1312678us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400037fd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:36.364797 (+ 364us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1261072128}
W20260812 19:07:36.379513 29283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.786s user 0.003s sys 0.010s
I20260812 19:07:36.370939 29578 rpcz_store.cc:276] 0812 19:07:33.760449 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.760574 (+ 125us) service_pool.cc:227] Handling call
0812 19:07:34.366752 (+606178us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:34.367037 (+ 285us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:35.162275 (+795238us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.362551 (+1200276us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b5400136354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:36.362892 (+ 341us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1786626816}
I20260812 19:07:36.368580 777 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.317713 711 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.369005 726 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.398701 730 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.399547 726 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.374583 693 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.391323 686 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.391604 700 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.366712 29450 rpcz_store.cc:276] 0812 19:07:35.175496 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.176026 (+ 530us) service_pool.cc:227] Handling call
0812 19:07:35.176363 (+ 337us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:35.176584 (+ 221us) raft_consensus.cc:1520] Early marking committed up to index 3
0812 19:07:36.137496 (+960912us) log.cc:842] Serialized 14 byte log entry
0812 19:07:36.361183 (+223687us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.361694 (+ 511us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:36.361796 (+ 102us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.361885 (+ 89us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.362140 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1005994,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":299,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":178,"thread_start_us":1125541,"threads_started":2}
I20260812 19:07:36.387588 821 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.395063 862 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:36.395562 30623 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:36.395995 446 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.396641 711 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.397331 728 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.399971 29452 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260812 19:07:36.390120 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 608039us
I20260812 19:07:36.259042 820 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.388772 647 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.389513 593 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.374827 546 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.398634 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:36.397697 709 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.394173 512 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:36.415211 29445 raft_consensus.cc:2396] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260812 19:07:36.413592 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.415799 446 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.416507 533 raft_consensus.cc:2781] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:36.417025 693 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.417215 709 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:36.425542 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 83660.4us
I20260812 19:07:36.461728 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.463573 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2195) took 1200 ms. Trace:
I20260812 19:07:36.475684 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:36.476552 30675 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:36.477169 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:36.477453 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.478269 30674 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:07:36.462384 29528 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.774s (SENT) [suppressed 3 similar messages]
I20260812 19:07:36.478950 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.491940 856 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.493029 593 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.492789 856 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:36.494419 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:36.494863 647 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.495754 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2067) took 1894 ms. Trace:
I20260812 19:07:36.496127 29580 rpcz_store.cc:276] 0812 19:07:34.601001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.686518 (+ 85517us) service_pool.cc:227] Handling call
0812 19:07:34.686854 (+ 336us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:34.811266 (+124412us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.811299 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:34.811338 (+ 39us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:34.811368 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.495274 (+1683906us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400135954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:36.495336 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.495566 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":285656832}
I20260812 19:07:36.497234 29445 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:07:36.498792 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.498128 820 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.471006 29331 rpcz_store.cc:276] 0812 19:07:35.224512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.227194 (+ 2682us) service_pool.cc:227] Handling call
0812 19:07:35.227561 (+ 367us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:35.227792 (+ 231us) raft_consensus.cc:1520] Early marking committed up to index 4
0812 19:07:36.263828 (+1036036us) log.cc:842] Serialized 14 byte log entry
0812 19:07:36.384787 (+120959us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.401425 (+ 16638us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:36.407115 (+ 5690us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.412697 (+ 5582us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.425268 (+ 12571us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":34148,"prepare.queue_time_us":952428,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":7875,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":14242,"thread_start_us":1035067,"threads_started":2}
I20260812 19:07:36.424948 533 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.498565 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.497579 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.504791 29572 raft_consensus.cc:2354] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4.
I20260812 19:07:36.502483 546 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:36.505302 29572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 485) took 1750 ms (client timeout 1951 ms). Trace:
W20260812 19:07:36.492933 29575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2062) took 1980 ms (client timeout 1857 ms). Trace:
W20260812 19:07:36.505674 29572 rpcz_store.cc:269] 0812 19:07:34.754988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.398178 (+1643190us) service_pool.cc:227] Handling call
0812 19:07:36.505238 (+107060us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.508546 820 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.508661 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.508905 546 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:36.509552 29572 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:36.510491 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.510609 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.511703 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2005) took 1912 ms. Trace:
I20260812 19:07:36.512130 29576 rpcz_store.cc:276] 0812 19:07:34.598745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.598866 (+ 121us) service_pool.cc:227] Handling call
0812 19:07:34.602245 (+ 3379us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:34.877181 (+274936us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.877213 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:36.103707 (+1226494us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b2c00032a08. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:36.104582 (+ 875us) log.cc:842] Serialized 26 byte log entry
0812 19:07:36.218549 (+113967us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:36.220901 (+ 2352us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.231052 (+ 10151us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:36.511282 (+280230us) raft_consensus.cc:1671] finished
0812 19:07:36.511309 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.511537 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":15725,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":55534,"spinlock_wait_cycles":45057792,"thread_start_us":383425,"threads_started":2,"wal-append.queue_time_us":300105}
I20260812 19:07:36.513753 446 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.481603 821 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.516290 29530 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:36.511790 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.512176 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.523147 761 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.523963 761 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.526307 761 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.409161 700 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.527124 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:36.501228 825 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:36.528064 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:36.478281 849 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.528498 709 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:36.528839 29456 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:36.418401 512 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:36.505859 29527 connection.cc:232] RPC call timeout handler was delayed by -1.1552070849575102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
I20260812 19:07:36.529233 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.538110 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.555711 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 482) took 1801 ms. Trace:
I20260812 19:07:36.554483 29451 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
W20260812 19:07:36.624796 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 77796.8us
W20260812 19:07:36.507133 29575 rpcz_store.cc:269] 0812 19:07:34.512601 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.512714 (+ 113us) service_pool.cc:227] Handling call
0812 19:07:36.492497 (+1979783us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54002003d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.492710 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":389190400}
I20260812 19:07:36.540522 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2006) took 1777 ms. Trace:
I20260812 19:07:36.705289 29576 rpcz_store.cc:276] 0812 19:07:34.753615 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.513747 (+1760132us) service_pool.cc:227] Handling call
0812 19:07:36.519291 (+ 5544us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.519443 (+ 152us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:36.519464 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.519497 (+ 33us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:36.530335 (+ 10838us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.530402 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.530507 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:36.702554 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 221073us
I20260812 19:07:36.514055 777 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:36.554786 730 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.673689 29574 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:36.556088 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 483) took 1801 ms. Trace:
I20260812 19:07:36.594844 643 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.701228 29572 rpcz_store.cc:276] 0812 19:07:34.754160 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.554595 (+1800435us) service_pool.cc:227] Handling call
0812 19:07:36.555046 (+ 451us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.555264 (+ 218us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:36.555287 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.555330 (+ 43us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:36.555357 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.555496 (+ 139us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.555635 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.506069 533 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:36.662520 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 229994us
I20260812 19:07:36.512259 528 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:36.650689 29580 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:36.693241 29332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2194) took 1468 ms (client timeout 1865 ms). Trace:
I20260812 19:07:36.602272 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2066) took 2010 ms. Trace:
I20260812 19:07:36.690387 821 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.734021 29578 rpcz_store.cc:276] 0812 19:07:34.754458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.555060 (+1800602us) service_pool.cc:227] Handling call
0812 19:07:36.555428 (+ 368us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.555745 (+ 317us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:36.555772 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.555831 (+ 59us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:36.555859 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.555904 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.555994 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.736022 29452 raft_consensus.cc:1226] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
W20260812 19:07:36.717547 29342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2088) took 1505 ms (client timeout 1862 ms). Trace:
I20260812 19:07:36.530653 647 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.739241 737 tablet_copy_client.cc:531] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 19:07:36.739566 30629 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:36.740375 30671 raft_consensus.cc:2354] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:36.826246 29567 rpcz_store.cc:276] 0812 19:07:34.591631 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.591855 (+ 224us) service_pool.cc:227] Handling call
0812 19:07:34.592110 (+ 255us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:34.949936 (+357826us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.949990 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:34.950033 (+ 43us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:34.950063 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.601779 (+1651716us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b5400177854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:36.601847 (+ 68us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.602075 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":822464256}
I20260812 19:07:36.540936 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.773705 29281 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:36.773931 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2007) took 2012 ms. Trace:
W20260812 19:07:36.827171 29332 rpcz_store.cc:269] 0812 19:07:35.224210 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.224320 (+ 110us) service_pool.cc:227] Handling call
0812 19:07:36.692770 (+1468450us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54003d8954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.693040 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":104026880}
W20260812 19:07:36.832473 29342 rpcz_store.cc:269] 0812 19:07:35.212099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.216875 (+ 4776us) service_pool.cc:227] Handling call
0812 19:07:36.717187 (+1500312us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54003778d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.717359 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":843699072}
I20260812 19:07:36.836272 29576 rpcz_store.cc:276] 0812 19:07:34.761081 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.745714 (+1984633us) service_pool.cc:227] Handling call
0812 19:07:36.773101 (+ 27387us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.773523 (+ 422us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:36.773548 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.773620 (+ 72us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:36.773657 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.773713 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.773837 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.773429 643 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:36.774888 849 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.736073 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.775607 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 487) took 2009 ms. Trace:
I20260812 19:07:36.776335 642 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:36.778179 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 45239.9us
I20260812 19:07:36.732091 29450 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:36.780994 700 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.781702 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:36.602283 776 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:36.648514 856 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.657586 686 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:36.603397 593 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.854157 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.854763 666 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.854553 593 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:36.855502 666 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:36.856637 593 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.546077 820 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.785403 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 749) took 2278 ms. Trace:
I20260812 19:07:36.859018 642 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.853829 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:36.862855 29579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 478) took 2282 ms (client timeout 2150 ms). Trace:
I20260812 19:07:36.858662 29339 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:36.863205 686 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.864391 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.785784 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.857664 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.787014 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.787148 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 605) took 2258 ms. Trace:
I20260812 19:07:36.785406 29527 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:36.813062 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 295160us
W20260812 19:07:36.611433 866 log.cc:924] Time spent T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.119s user 0.000s sys 0.000s
I20260812 19:07:36.574734 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.738260 29574 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:36.834107 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:36.648604 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1827) took 1493 ms. Trace:
I20260812 19:07:36.688926 771 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:36.840521 29572 rpcz_store.cc:276] 0812 19:07:34.766098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.770109 (+2004011us) service_pool.cc:227] Handling call
0812 19:07:36.774899 (+ 4790us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.775254 (+ 355us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:36.775280 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:36.775326 (+ 46us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:36.775397 (+ 71us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.775445 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.775550 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:36.842716 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:36.845116 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:36.845657 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 106204us
I20260812 19:07:36.771294 726 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.743366 825 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:36.853842 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:36.857385 820 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:36.852051 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:36.866886 29331 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:37.538460 29326 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:36.934130 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.540112 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.848989 693 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:36.980178 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:37.541361 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 822855us
I20260812 19:07:37.541543 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:37.537168 30675 consensus_meta.cc:287] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.060s user 0.013s sys 0.000s
I20260812 19:07:37.543035 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.536917 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:36.866775 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.132191 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.536620 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:36.902854 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 66845.6us
I20260812 19:07:37.540481 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.398586 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.356462 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:36.858134 30685 rpcz_store.cc:276] 0812 19:07:34.507039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.510262 (+ 3223us) service_pool.cc:227] Handling call
0812 19:07:34.510527 (+ 265us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:34.596738 (+ 86211us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:34.596769 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:34.596808 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:34.596839 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:36.784932 (+2188093us) spinlock_profiling.cc:243] Waited 86 ms on lock 0x7b54004b05d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:36.784997 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.785188 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":197845760}
I20260812 19:07:36.864707 593 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:37.255789 778 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260812 19:07:37.005473 30679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 746) took 2498 ms (client timeout 1926 ms). Trace:
I20260812 19:07:36.952837 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:37.553341 593 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:36.852367 728 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.933324 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:36.989439 647 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:36.863312 29579 rpcz_store.cc:269] 0812 19:07:34.580440 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.580689 (+ 249us) service_pool.cc:227] Handling call
0812 19:07:36.862472 (+2281783us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400172d54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.862633 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3882302336}
I20260812 19:07:37.553071 29580 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:37.550390 29576 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
W20260812 19:07:37.553598 30679 rpcz_store.cc:269] 0812 19:07:34.501779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.504243 (+ 2464us) service_pool.cc:227] Handling call
0812 19:07:36.714602 (+2210359us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:36.999998 (+285396us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12684544}
W20260812 19:07:37.556383 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 300892us
I20260812 19:07:37.546684 29458 rpcz_store.cc:276] 0812 19:07:35.155237 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:35.155359 (+ 122us) service_pool.cc:227] Handling call
0812 19:07:35.595575 (+440216us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:35.595778 (+ 203us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:35.595803 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:35.595843 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:35.595881 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:35.595933 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:36.648209 (+1052276us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54003f3c50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.648413 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1005611776}
I20260812 19:07:36.866765 29453 rpcz_store.cc:276] 0812 19:07:34.529076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.752032 (+222956us) service_pool.cc:227] Handling call
0812 19:07:36.786757 (+2034725us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5400213254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:36.786960 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":51054976}
I20260812 19:07:37.555159 29335 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
W20260812 19:07:37.274551 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 479) took 2674 ms (client timeout 3230 ms). Trace:
W20260812 19:07:37.461047 857 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.478s user 0.000s sys 0.001s
I20260812 19:07:36.905997 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.555855 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 486) took 2790 ms. Trace:
I20260812 19:07:37.408648 849 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.562928 29575 rpcz_store.cc:276] 0812 19:07:34.765754 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.740730 (+1974976us) service_pool.cc:227] Handling call
0812 19:07:36.741221 (+ 491us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.553742 (+812521us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:37.553774 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:37.553815 (+ 41us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:37.553850 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.555594 (+ 1744us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b5400178754. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:37.555636 (+ 42us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:37.555762 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1832496000}
I20260812 19:07:37.570627 29457 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:37.552985 778 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:36.866436 711 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:36.849361 862 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:07:37.575157 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 619) took 1104 ms. Trace:
I20260812 19:07:37.575686 29569 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:37.578887 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2013) took 1307 ms. Trace:
W20260812 19:07:37.581061 30623 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.725s user 0.012s sys 0.039s
I20260812 19:07:36.956226 706 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 19:07:37.561510 29577 rpcz_store.cc:269] 0812 19:07:34.600555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.600689 (+ 134us) service_pool.cc:227] Handling call
0812 19:07:37.274173 (+2673484us) spinlock_profiling.cc:243] Waited 93.1 ms on lock 0x7b540009f454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:37.274370 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":214023552}
I20260812 19:07:37.439553 643 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.857658 666 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:36.861882 642 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.580675 29457 rpcz_store.cc:276] 0812 19:07:36.483087 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.833768 (+350681us) service_pool.cc:227] Handling call
0812 19:07:37.575094 (+741326us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:37.556092 645 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 19:07:37.581825 30623 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.726s user 0.012s sys 0.040s
I20260812 19:07:37.287039 777 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:37.556821 30670 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:37.555728 593 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.581449 29579 rpcz_store.cc:276] 0812 19:07:36.260488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:37.556324 (+1295836us) service_pool.cc:227] Handling call
0812 19:07:37.565122 (+ 8798us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.565350 (+ 228us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:37.565373 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:37.565411 (+ 38us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:37.565442 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.568094 (+ 2652us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:37.568220 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:37.537566 825 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:37.593106 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 737208us
W20260812 19:07:37.582975 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 797668us
I20260812 19:07:37.598479 30629 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:37.478096 713 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:37.574589 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2008) took 2802 ms. Trace:
I20260812 19:07:37.580510 778 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:37.608354 30629 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:37.574374 726 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:37.600579 673 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:37.542516 771 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.540377 29578 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:37.607213 713 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:37.608537 745 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:36.858760 528 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:37.576330 512 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.611501 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 488) took 1272 ms. Trace:
I20260812 19:07:37.617473 29567 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:37.580256 799 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:37.596805 686 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.618549 772 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:36.886104 709 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.717s
I20260812 19:07:36.932147 776 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:37.605856 29574 rpcz_store.cc:276] 0812 19:07:34.763518 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.737169 (+1973651us) service_pool.cc:227] Handling call
0812 19:07:36.737748 (+ 579us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.543541 (+805793us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.565901 (+ 22360us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:37.629522 820 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.588657 728 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260812 19:07:37.630328 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.633008 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.630110 745 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:37.655678 745 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:37.656091 879 log.cc:924] Time spent T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
W20260812 19:07:37.656844 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 56451.1us
I20260812 19:07:37.634552 29576 rpcz_store.cc:276] 0812 19:07:36.338889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:37.558150 (+1219261us) service_pool.cc:227] Handling call
0812 19:07:37.600877 (+ 42727us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.611179 (+ 10302us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:37.611202 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:37.611241 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:37.611286 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.611339 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:37.611448 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:37.635753 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:37.639619 771 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.641224 825 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:37.612445 30675 raft_consensus.cc:2396] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:37.646868 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:37.647576 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2012) took 1378 ms. Trace:
I20260812 19:07:37.645046 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.662865 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.667128 29567 rpcz_store.cc:276] 0812 19:07:36.260063 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:37.542735 (+1282672us) service_pool.cc:227] Handling call
0812 19:07:37.576217 (+ 33482us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.634832 (+ 58615us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.638216 (+ 3384us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:37.669430 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 94491.1us
I20260812 19:07:37.645382 772 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:37.643514 673 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:37.653661 726 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:37.662699 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.671670 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:37.661335 708 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 2.832s user 0.028s sys 0.042s
I20260812 19:07:37.675879 29580 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:37.676990 30685 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:37.686628 29341 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:37.680485 29342 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:37.665392 647 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.690964 29335 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:37.688628 30629 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:37.671502 862 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:37.649065 711 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:07:37.679277 708 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 2.910s user 0.028s sys 0.042s
I20260812 19:07:37.672770 772 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:37.686942 673 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:37.698467 872 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.793s user 0.015s sys 0.022s
W20260812 19:07:37.705170 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 99585.9us
I20260812 19:07:37.676283 713 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.711027 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.665381 799 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.709103 30629 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:37.688925 30675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 733) took 1213 ms (client timeout 1501 ms). Trace:
I20260812 19:07:37.719755 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.724252 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.724998 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.729074 30629 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:37.723249 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:37.716507 872 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.852s user 0.015s sys 0.023s
I20260812 19:07:37.733081 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.705862 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:37.741207 29567 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:37.747637 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.705458 811 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:37.722257 862 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:37.695050 726 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:37.753692 29338 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.773s user 0.025s sys 0.050s
W20260812 19:07:37.757982 29338 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.897s user 0.025s sys 0.050s
I20260812 19:07:37.733170 686 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:37.760372 30684 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:37.762420 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 587) took 1416 ms. Trace:
I20260812 19:07:37.762821 29338 rpcz_store.cc:276] 0812 19:07:36.346041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.558490 (+212449us) service_pool.cc:227] Handling call
0812 19:07:36.604936 (+ 46446us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:36.683255 (+ 78319us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:36.835595 (+152340us) log.cc:842] Serialized 14 byte log entry
0812 19:07:37.762049 (+926454us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:37.762117 (+ 68us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:37.762173 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:37.762234 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:37.762360 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":160214,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":896953,"threads_started":1}
I20260812 19:07:37.764573 702 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:37.765038 702 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:37.765395 702 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:37.769375 799 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.770781 776 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.770603 811 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.742918 711 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:37.773003 811 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.773164 711 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.774011 528 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:37.775952 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.754570 693 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:37.777994 630 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s user 0.008s sys 0.034s
W20260812 19:07:37.778434 630 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s user 0.008s sys 0.034s
I20260812 19:07:37.779300 862 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:37.779798 862 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:37.729790 30675 rpcz_store.cc:269] 0812 19:07:36.467448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.475220 (+ 7772us) service_pool.cc:227] Handling call
0812 19:07:37.677295 (+1202075us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:37.782297 686 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:37.788476 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:37.789476 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.794239 737 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Bootstrap starting.
I20260812 19:07:37.794126 30625 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:37.804296 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 94247.1us
I20260812 19:07:37.805306 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:37.782692 693 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:37.788081 686 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:37.812335 29530 connection.cc:232] RPC call timeout handler was delayed by -1.2371064210310578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260812 19:07:37.808320 693 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:37.826552 778 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.829066 512 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:37.829859 512 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.830629 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:37.831436 642 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:37.831969 642 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.832746 512 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.832962 666 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:37.833528 666 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.833345 512 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:37.834133 642 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.842267 726 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:37.847297 666 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:37.833945 643 consensus_queue.cc:1046] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:37.810906 849 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:37.819680 647 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:37.828788 778 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:37.868587 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.838797 709 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:37.874316 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.874707 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.874971 29567 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:37.875517 29580 raft_consensus.cc:2379] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:37.875723 29567 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57278
I20260812 19:07:37.876307 29567 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:37.876614 29567 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:37.878093 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.878475 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.879594 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.869622 711 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:37.882380 29527 leader_election.cc:398] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:37.882776 29527 leader_election.cc:401] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260812 19:07:37.882475 711 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.884253 29572 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:07:37.884620 711 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:37.885144 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 521995us
I20260812 19:07:37.885843 29282 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:37.887454 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.888283 29576 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:37.890522 799 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:37.890925 799 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:37.891301 799 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:37.892082 533 consensus_meta.cc:287] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.050s user 0.007s sys 0.004s
I20260812 19:07:37.892525 533 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.893707 30684 raft_consensus.cc:1226] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:37.885576 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.889607 29282 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:37.845890 30676 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:37.878777 669 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260812 19:07:37.877269 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.833582 593 consensus_queue.cc:1046] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260812 19:07:37.914336 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.911806 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.912464 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.917172 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.907897 630 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:37.918373 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:37.877081 702 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.925833 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:37.905500 528 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:37.981801 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:37.979600 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 52726.3us
W20260812 19:07:37.979033 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 896153us
W20260812 19:07:37.978415 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 57818.9us
I20260812 19:07:37.898276 647 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:37.915277 29574 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:38.019953 898 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:38.020495 898 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:38.020897 898 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:38.022145 666 raft_consensus.cc:3032] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:38.022557 666 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:38.023419 666 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:37.998308 630 raft_consensus.cc:686] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:38.020274 639 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
W20260812 19:07:38.032512 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 303517us
W20260812 19:07:38.036715 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 203620us
I20260812 19:07:38.052534 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.054842 706 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.055873 713 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.055605 706 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.056483 713 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:38.060720 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 186031us
I20260812 19:07:38.065737 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.066570 29335 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.065737 30625 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.079713 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2011) took 1820 ms. Trace:
I20260812 19:07:38.080112 29578 rpcz_store.cc:276] 0812 19:07:36.259621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:37.539635 (+1280014us) service_pool.cc:227] Handling call
0812 19:07:37.539957 (+ 322us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.540198 (+ 241us) raft_consensus.cc:1520] Early marking committed up to index 3
0812 19:07:37.724242 (+184044us) log.cc:842] Serialized 14 byte log entry
0812 19:07:38.074510 (+350268us) log.cc:842] Serialized 14 byte log entry
0812 19:07:38.079389 (+ 4879us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:38.079447 (+ 58us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:38.079488 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:38.079548 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:38.079658 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6664,"prepare.queue_time_us":27893,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":8204,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":2918,"thread_start_us":318181,"threads_started":1}
I20260812 19:07:38.083928 666 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.084946 647 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.089110 29335 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:38.094959 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.099948 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.103199 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.107522 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.108863 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.111305 686 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.113174 642 raft_consensus.cc:3032] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:38.113615 642 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:38.113989 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:38.115433 763 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.554s user 0.032s sys 0.030s
W20260812 19:07:38.115793 763 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.630s user 0.033s sys 0.031s
I20260812 19:07:38.044462 772 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.064092 533 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:38.118758 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 53116.1us
I20260812 19:07:38.120075 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.121472 29530 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.127229 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.127900 29457 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:38.113417 29567 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:38.120668 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 266024us
I20260812 19:07:38.132320 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.134555 29567 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:38.132475 30623 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.137454 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:38.138610 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 83178us
I20260812 19:07:38.140668 29453 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:07:38.142481 30681 raft_consensus.cc:2379] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 2: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:07:38.142212 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 61234.3us
I20260812 19:07:38.123198 702 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.125689 639 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.126330 669 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:38.154497 29530 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.113554 745 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.157744 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.157579 669 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.163501 904 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:38.078648 630 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.163993 728 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.164789 693 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.129385 30670 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:38.167460 29338 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.141582 905 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:38.177074 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 84739.4us
I20260812 19:07:38.168533 669 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:38.184309 904 raft_consensus.cc:686] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:38.188982 904 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.189924 713 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.166188 686 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:38.123391 862 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.193351 29530 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.178167 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 735) took 1709 ms. Trace:
I20260812 19:07:38.201570 30670 rpcz_store.cc:276] 0812 19:07:36.812802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.836620 (+ 23818us) service_pool.cc:227] Handling call
0812 19:07:38.174869 (+1338249us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:38.202256 664 raft_consensus.cc:2741] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260812 19:07:38.203332 29530 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.182374 917 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260812 19:07:38.209764 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 52085.9us
I20260812 19:07:38.185549 706 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.199752 777 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:38.214332 30629 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:38.217542 664 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.223097 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.220966 29527 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.228824 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.229197 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.230293 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.231752 30686 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:38.231837 686 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:38.232363 686 raft_consensus.cc:686] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.232991 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.233258 686 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.238888 29335 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:38.243137 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.245198 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:38.245558 30625 connection.cc:232] RPC call timeout handler was delayed by -1.0388000740204006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
I20260812 19:07:38.248915 811 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:38.249392 811 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.238054 825 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260812 19:07:38.202411 905 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.257447 772 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.258955 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.213614 917 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:38.202919 642 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.277980 29530 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.246392 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.257778 528 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:38.219322 777 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:38.244325 30676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 761) took 1399 ms (client timeout 1531 ms). Trace:
I20260812 19:07:38.248036 29399 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.261487 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.270470 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.284813 29331 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:07:38.278898 29339 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:38.286161 645 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.286327 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.286675 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.220157 799 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.287750 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.283439 777 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.288923 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.289368 799 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.290544 811 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.292359 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.288851 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1843) took 1791 ms. Trace:
I20260812 19:07:38.293555 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.293234 29445 rpcz_store.cc:276] 0812 19:07:36.497534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.534929 (+ 37395us) service_pool.cc:227] Handling call
0812 19:07:36.864428 (+329499us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:36.864612 (+ 184us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:37.555588 (+690976us) raft_consensus.cc:1520] Early marking committed up to index 4
0812 19:07:37.980707 (+425119us) log.cc:842] Serialized 14 byte log entry
0812 19:07:38.117377 (+136670us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:38.117493 (+ 116us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:38.117688 (+ 195us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:38.288616 (+170928us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54001aca54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:38.288671 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:38.288796 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16013,"prepare.queue_time_us":488620,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":16233,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7231,"spinlock_wait_cycles":1604091904,"thread_start_us":508333,"threads_started":2,"wal-append.queue_time_us":121094}
W20260812 19:07:38.295591 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 154530us
I20260812 19:07:38.287657 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.237182 898 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.311177 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:38.315156 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 53148.9us
I20260812 19:07:38.316042 29530 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.313122 29337 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260812 19:07:38.313803 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 65849us
I20260812 19:07:38.313980 30685 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:38.287533 897 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.266s user 0.000s sys 0.004s
I20260812 19:07:38.280773 917 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:38.271695 763 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.349718 664 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:38.350500 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:38.353786 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 57572.2us
W20260812 19:07:38.353463 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 109548us
I20260812 19:07:38.362815 30681 raft_consensus.cc:2396] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:38.389211 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:38.283288 30676 rpcz_store.cc:269] 0812 19:07:36.842764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:36.851135 (+ 8371us) service_pool.cc:227] Handling call
0812 19:07:38.225697 (+1374562us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b5400765f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:38.238855 (+ 13158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1685963648}
I20260812 19:07:38.301849 528 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:07:38.394289 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 78340.6us
I20260812 19:07:38.401036 29530 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.401271 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:38.391759 29338 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:38.396277 29342 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260812 19:07:38.397608 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.396497 30674 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:38.403642 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.403923 30629 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:38.335148 693 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.404253 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.405812 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.407722 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:38.405211 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.409507 29527 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.410049 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.398226 30679 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:38.352800 669 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.404527 29338 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:38.442633 669 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.411458 708 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:38.447021 898 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.447271 30670 raft_consensus.cc:1226] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:38.412938 763 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.448398 30686 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:38.440133 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.449558 763 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.429674 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.429804 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.455279 29579 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.430132 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.456220 29341 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.436028 29335 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:38.430455 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:07:38.404812 872 log.cc:924] Time spent T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Append to log took a long time: real 0.065s user 0.001s sys 0.000s
I20260812 19:07:38.390678 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:38.437837 926 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.085s user 0.000s sys 0.000s
I20260812 19:07:38.437647 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.439358 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.440182 728 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.449726 29580 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.450503 666 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:38.451028 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.488412 29281 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.388854 593 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:07:38.428840 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.400601 664 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260812 19:07:38.476133 811 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:38.428813 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:38.453666 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49837.9us
I20260812 19:07:38.456084 29338 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
W20260812 19:07:38.469622 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 259144us
I20260812 19:07:38.474237 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.484709 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.494020 29337 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:38.485388 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.486234 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.489388 905 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.490170 763 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:38.498306 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:38.498857 29338 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:38.475317 825 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260812 19:07:38.491021 811 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.490370 29574 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:38.504709 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.509644 30629 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:38.491276 666 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:38.514503 666 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.515930 30675 raft_consensus.cc:2379] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 2: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:38.516606 29174 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55520 (request call id 374) took 4690 ms. Trace:
I20260812 19:07:38.500464 593 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:38.518576 929 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:38.518496 29341 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:38.518934 593 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:38.518519 30676 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:38.519426 593 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:38.520126 30670 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260812 19:07:38.505353 664 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:38.521224 664 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:38.504688 645 consensus_queue.cc:1046] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260812 19:07:38.521744 728 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:38.490428 706 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260812 19:07:38.522290 645 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:38.522142 664 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.523277 728 raft_consensus.cc:3032] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:38.523952 929 raft_consensus.cc:686] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.523681 728 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:38.524739 811 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.524941 929 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.525581 862 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.505062 934 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:38.525359 728 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.527068 728 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:38.528911 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.517683 29579 raft_consensus.cc:2454] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:38.546056 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.546934 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.549705 29580 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:38.551641 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.552393 30629 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:38.430388 528 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:38.558322 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 92073.6us
I20260812 19:07:38.553072 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.555732 29579 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:38.556804 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:38.523391 645 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:38.526551 934 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.560627 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.557988 29577 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:38.532114 904 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.566382 30684 raft_consensus.cc:1226] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.571352 29335 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.523396 664 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.570478 29337 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:38.579445 904 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:38.580227 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:07:38.580607 933 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.097s user 0.000s sys 0.000s
I20260812 19:07:38.581624 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.584702 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.585870 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.590229 29577 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.592648 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.593466 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.517004 29174 rpcz_store.cc:276] 0812 19:07:33.800644 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.800799 (+ 155us) service_pool.cc:227] Handling call
0812 19:07:38.483934 (+4683135us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:36.260242 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:36.261177 (+ 935us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:36.261248 (+ 71us) write_op.cc:435] Acquired schema lock
0812 19:07:36.261283 (+ 35us) tablet.cc:660] Decoding operations
0812 19:07:36.261803 (+ 520us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:36.261900 (+ 97us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:36.261986 (+ 86us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:36.262425 (+ 439us) tablet.cc:699] Row locks acquired
0812 19:07:36.262469 (+ 44us) write_op.cc:260] PREPARE: finished
0812 19:07:36.262765 (+ 296us) write_op.cc:270] Start()
0812 19:07:36.262933 (+ 168us) write_op.cc:276] Timestamp: P: 1786561656262692 usec, L: 0
0812 19:07:36.262976 (+ 43us) op_driver.cc:331] REPLICATION: starting
0812 19:07:36.292865 (+ 29889us) log.cc:842] Serialized 541 byte log entry
0812 19:07:36.626300 (+333435us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b5400002dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0a198 00007f9c6d81dade 00007f9c6d81d2b8 00007f9c6d821ba1 00007f9c6d821b59 00007f9c6d821ae9 00007f9c6d821ab1 00007f9c6d820dad 00007f9c7014a204 00007f9c7014a039
0812 19:07:37.926394 (+1300094us) op_driver.cc:444] REPLICATION: finished
0812 19:07:38.217761 (+291367us) write_op.cc:301] APPLY: starting
0812 19:07:38.217838 (+ 77us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:38.217979 (+ 141us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:38.218013 (+ 34us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:38.218363 (+ 350us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:38.218636 (+ 273us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:38.227164 (+ 8528us) write_op.cc:312] APPLY: finished
0812 19:07:38.291819 (+ 64655us) log.cc:842] Serialized 23 byte log entry
0812 19:07:38.292046 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:38.292118 (+ 72us) write_op.cc:454] Released schema lock
0812 19:07:38.292169 (+ 51us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:38.292514 (+ 345us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:38.292542 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1062578,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":214270,"apply.run_cpu_time_us":2433,"apply.run_wall_time_us":75043,"num_ops":1,"prepare.queue_time_us":967086,"prepare.run_cpu_time_us":36453,"prepare.run_wall_time_us":366364,"raft.queue_time_us":1434653,"raft.run_cpu_time_us":35610,"raft.run_wall_time_us":361441,"replication_time_us":1646859,"spinlock_wait_cycles":35666048,"thread_start_us":2157509,"threads_started":4,"wal-append.queue_time_us":482703}]]}
I20260812 19:07:38.594118 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.598351 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.617102 29332 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.599197 29334 raft_consensus.cc:1226] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.599308 945 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:38.617662 945 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.618100 945 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.599109 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.619812 30675 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:38.620756 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.621351 30675 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 1.
I20260812 19:07:38.599835 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.622208 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:38.623724 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:38.624898 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.626222 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:38.626314 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.626792 29450 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:38.627158 30623 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.627868 29281 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.628803 30623 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.600456 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:38.600586 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.572259 934 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.560051 643 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260812 19:07:38.602468 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.599433 29331 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:38.572610 533 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:38.562095 645 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:38.626366 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:38.604215 904 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:38.600198 29530 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.629710 29458 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:38.626971 29446 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:38.553197 693 consensus_queue.cc:1046] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260812 19:07:38.651351 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:38.659698 29457 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.651276 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.654619 30679 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.654639 645 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.666702 29339 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:07:38.671669 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 114957us
I20260812 19:07:38.640324 898 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.640103 938 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:38.689061 745 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:38.524093 777 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:38.661085 904 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.693516 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55498 (request call id 383) took 4179 ms. Trace:
I20260812 19:07:38.672629 771 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:38.682299 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:38.742449 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 91912.1us
I20260812 19:07:38.744534 29282 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:38.743328 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 114453us
W20260812 19:07:38.746724 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 146605us
I20260812 19:07:38.749789 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.750103 29530 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.751652 29530 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.754961 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.691511 938 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.762720 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:38.763499 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 134743us
I20260812 19:07:38.765749 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.768246 30623 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.764765 771 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:38.769196 938 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:38.769622 771 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:38.771173 771 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:38.771260 925 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.771585 771 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.771036 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.765127 713 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:38.773274 713 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.773643 713 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:38.770131 849 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:38.773890 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.774446 776 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.775051 686 raft_consensus.cc:2781] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:38.775210 630 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.775579 686 raft_consensus.cc:686] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.776455 686 consensus_queue.cc:237] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.778676 706 consensus_queue.cc:1046] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:38.779886 763 raft_consensus.cc:3032] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:38.779721 917 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.780355 763 raft_consensus.cc:729] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:38.763025 708 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.781303 763 consensus_queue.cc:260] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.765596 711 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:38.782331 711 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.781663 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:38.782752 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.783432 693 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:38.783720 925 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:38.783931 693 raft_consensus.cc:686] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.784875 693 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.785547 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.786247 925 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:38.786597 925 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.786914 925 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.787642 29282 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.788038 29281 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.788048 30623 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:38.788447 29282 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.743655 645 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:38.791277 29399 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:38.793444 29335 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:38.804965 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:38.782711 711 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:07:38.811697 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 64349.3us
I20260812 19:07:38.814025 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:38.696070 29176 rpcz_store.cc:276] 0812 19:07:34.511042 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:34.511300 (+ 258us) service_pool.cc:227] Handling call
0812 19:07:38.690576 (+4179276us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:36.690135 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:36.690916 (+ 781us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:36.690965 (+ 49us) write_op.cc:435] Acquired schema lock
0812 19:07:36.690979 (+ 14us) tablet.cc:660] Decoding operations
0812 19:07:36.691424 (+ 445us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:36.691490 (+ 66us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:36.691520 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0812 19:07:36.691734 (+ 214us) tablet.cc:699] Row locks acquired
0812 19:07:36.691749 (+ 15us) write_op.cc:260] PREPARE: finished
0812 19:07:36.691966 (+ 217us) write_op.cc:270] Start()
0812 19:07:36.692095 (+ 129us) write_op.cc:276] Timestamp: P: 1786561656691910 usec, L: 0
0812 19:07:36.692116 (+ 21us) op_driver.cc:331] REPLICATION: starting
0812 19:07:36.692982 (+ 866us) log.cc:842] Serialized 541 byte log entry
0812 19:07:38.170866 (+1477884us) op_driver.cc:444] REPLICATION: finished
0812 19:07:38.292971 (+122105us) write_op.cc:301] APPLY: starting
0812 19:07:38.293053 (+ 82us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:38.293138 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:38.355566 (+ 62428us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:38.413918 (+ 58352us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:38.450490 (+ 36572us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0812 19:07:38.450546 (+ 56us) write_op.cc:312] APPLY: finished
0812 19:07:38.450767 (+ 221us) log.cc:842] Serialized 23 byte log entry
0812 19:07:38.450953 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:38.451041 (+ 88us) write_op.cc:454] Released schema lock
0812 19:07:38.451084 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:38.451475 (+ 391us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:38.451508 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":723898,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":117405,"apply.run_cpu_time_us":2552,"apply.run_wall_time_us":158784,"num_ops":1,"prepare.queue_time_us":1023769,"prepare.run_cpu_time_us":3171,"prepare.run_wall_time_us":3171,"replication_time_us":1476616,"thread_start_us":84128,"threads_started":1}]]}
I20260812 19:07:38.801829 737 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260812 19:07:38.792327 811 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:38.833487 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.792492 799 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:38.832351 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:38.807132 898 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260812 19:07:38.842252 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 78414.5us
I20260812 19:07:38.789981 777 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:38.845880 29338 raft_consensus.cc:2379] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:38.846493 29567 raft_consensus.cc:1226] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.836087 947 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:38.837750 533 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:38.877893 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 122886us
I20260812 19:07:38.847319 811 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:38.783963 945 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:38.884553 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 70626.9us
I20260812 19:07:38.858484 898 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:38.845525 799 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:38.838155 776 consensus_queue.cc:1046] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260812 19:07:38.886299 30679 raft_consensus.cc:3032] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:38.886498 936 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:38.885212 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.878845 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.896901 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.879482 947 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:38.898147 947 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:38.883412 811 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:38.904034 29335 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:38.905279 29403 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.894348 799 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:38.912778 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:38.914026 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:38.914964 932 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:38.915421 932 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:38.908756 825 consensus_queue.cc:1046] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260812 19:07:38.916154 948 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.892246 898 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:38.924226 771 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.923308 945 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:38.926208 30676 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:38.927206 593 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:38.927950 726 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:38.926102 737 tablet_bootstrap.cc:492] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Bootstrap complete.
I20260812 19:07:38.894097 905 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:38.922431 958 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:38.934427 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:38.934682 771 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:38.935012 924 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:38.941025 29579 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:38.895625 30679 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.905978 777 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:38.954998 29403 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:38.915777 643 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:38.915782 932 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:38.973049 29337 raft_consensus.cc:1226] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.951470 728 consensus_queue.cc:1046] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260812 19:07:38.928421 666 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:38.956676 726 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:38.993914 29338 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:38.964143 905 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:38.994653 29450 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:38.961804 737 ts_tablet_manager.cc:1397] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Time spent bootstrapping tablet: real 1.168s user 0.026s sys 0.024s
I20260812 19:07:38.973601 924 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:38.967191 776 raft_consensus.cc:3032] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:38.957173 825 consensus_queue.cc:1046] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260812 19:07:38.967064 958 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:39.010610 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:39.011777 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:38.961891 711 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.015074 29342 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.010206 536 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.014284 29403 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.003036 905 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:39.008846 776 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:39.006812 948 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:39.009176 924 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:39.007149 917 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.021325 30629 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.014384 958 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:38.976716 947 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.022572 825 consensus_queue.cc:1225] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260812 19:07:39.014091 945 consensus_queue.cc:1046] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260812 19:07:38.981088 533 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.052404 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 89819.3us
I20260812 19:07:39.052304 29459 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:39.054032 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:39.050338 964 log.cc:924] Time spent T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260812 19:07:39.046203 772 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.047451 849 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:39.051771 29580 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:39.051757 593 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.103857 29567 raft_consensus.cc:1226] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:39.100436 29332 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:39.108793 29334 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
W20260812 19:07:39.111836 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 215529us
I20260812 19:07:39.112597 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.050520 917 consensus_queue.cc:1046] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260812 19:07:39.038187 30679 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.118604 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:39.118769 30679 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:39.123608 29342 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.124486 29338 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:39.127179 917 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:39.127573 917 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:39.128480 919 log.cc:924] Time spent T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.146s user 0.001s sys 0.001s
I20260812 19:07:39.038290 726 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.117086 986 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.136s user 0.001s sys 0.000s
W20260812 19:07:39.132846 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 257982us
I20260812 19:07:39.140017 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:39.140797 29450 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.142167 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.147132 29530 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:39.116225 849 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:39.188669 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.189683 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:39.106390 938 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.188699 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:39.199627 29576 raft_consensus.cc:3032] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:39.128715 917 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.222100 29576 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:39.223241 948 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:07:39.200314 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 58260.9us
I20260812 19:07:39.202445 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.201139 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:39.226562 29403 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.181303 536 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
W20260812 19:07:39.252394 29457 consensus_meta.cc:287] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.555s user 0.015s sys 0.007s
I20260812 19:07:39.252971 29457 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:39.189903 666 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:39.257115 29528 leader_election.cc:334] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.652s (SENT)
W20260812 19:07:39.257337 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 207277us
I20260812 19:07:39.257587 29339 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:39.134248 862 consensus_queue.cc:1046] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260812 19:07:39.142584 799 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.147783 709 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260812 19:07:39.234784 711 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.255602 643 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.152379 969 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260812 19:07:39.263190 30629 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:39.258901 970 log.cc:924] Time spent T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260812 19:07:39.265439 917 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:39.277724 536 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:39.305325 1012 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 19:07:39.306600 989 log.cc:924] Time spent T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260812 19:07:39.306336 536 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.318871 912 raft_consensus.cc:3032] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:39.289165 947 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:39.319310 912 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:39.320199 912 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.321489 979 log.cc:924] Time spent T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.180s user 0.001s sys 0.000s
I20260812 19:07:39.225354 948 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.312188 849 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.326136 709 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:39.327464 709 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:39.328369 763 raft_consensus.cc:3037] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:39.337087 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 68747.4us
I20260812 19:07:39.337914 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.274282 981 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:39.325698 29576 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.377795 29528 leader_election.cc:334] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.774s (SENT)
W20260812 19:07:39.364838 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 220837us
W20260812 19:07:39.403427 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 91424.6us
I20260812 19:07:39.403805 29576 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:39.409188 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 209982us
I20260812 19:07:39.303021 898 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.398707 934 consensus_meta.cc:287] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.742s user 0.003s sys 0.019s
W20260812 19:07:39.413373 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 75548.5us
I20260812 19:07:39.365125 1000 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:39.328670 709 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260812 19:07:39.410751 981 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:39.447742 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:39.423993 925 consensus_meta.cc:287] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.633s user 0.011s sys 0.000s
I20260812 19:07:39.456125 925 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.450143 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 220306us
I20260812 19:07:39.506127 29403 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.506502 29282 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:39.452078 849 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:39.509981 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 62304.5us
I20260812 19:07:39.511324 799 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.508106 925 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:39.519850 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.527065 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.421412 934 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.529595 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 747) took 1294 ms. Trace:
I20260812 19:07:39.499673 938 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:39.530123 30685 rpcz_store.cc:276] 0812 19:07:38.229900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.238917 (+ 9017us) service_pool.cc:227] Handling call
0812 19:07:38.311680 (+ 72763us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:38.353087 (+ 41407us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:39.523908 (+1170821us) spinlock_profiling.cc:243] Waited 58.8 ms on lock 0x7b540075dd50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:39.524041 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":135265024}
W20260812 19:07:39.544500 29528 leader_election.cc:334] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.865s (SENT)
I20260812 19:07:39.546241 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:39.547043 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.518124 997 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.524160 1000 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:39.553678 29341 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:39.524573 947 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:39.554843 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.559473 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.561203 1024 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.575523 1010 consensus_queue.cc:1046] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260812 19:07:39.575837 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.578137 771 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:39.531649 849 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.583353 849 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:39.588953 29576 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260812 19:07:39.612473 1013 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
I20260812 19:07:39.612725 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.613480 922 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260812 19:07:39.458726 737 raft_consensus.cc:348] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 NON_PARTICIPANT]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:39.564239 811 consensus_meta.cc:287] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.632s user 0.006s sys 0.005s
W20260812 19:07:39.562956 713 consensus_meta.cc:287] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.760s user 0.006s sys 0.008s
I20260812 19:07:39.662622 29335 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:39.533288 1023 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.566437 1000 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 19:07:39.601150 932 consensus_meta.cc:287] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.592s user 0.006s sys 0.003s
W20260812 19:07:39.670781 29528 leader_election.cc:334] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.991s (SENT)
I20260812 19:07:39.671254 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:39.673605 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:39.621340 997 raft_consensus.cc:3032] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:39.683847 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:39.667196 917 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:39.687172 771 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 19:07:39.691529 1030 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.130s user 0.000s sys 0.000s
I20260812 19:07:39.691823 980 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
W20260812 19:07:39.695272 905 consensus_meta.cc:287] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.657s user 0.010s sys 0.000s
I20260812 19:07:39.706633 980 raft_consensus.cc:686] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:39.719206 934 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.721618 912 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:39.709029 30679 consensus_meta.cc:287] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.590s user 0.006s sys 0.002s
I20260812 19:07:39.683568 997 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:39.684804 898 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:39.699589 924 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.649s user 0.001s sys 0.009s
I20260812 19:07:39.590111 981 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.730639 737 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Initialized, Role: LEARNER
I20260812 19:07:39.712771 917 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.732482 686 raft_consensus.cc:3032] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:39.666707 713 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.668166 811 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.741858 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 194862us
W20260812 19:07:39.735128 904 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.993s user 0.007s sys 0.004s
I20260812 19:07:39.740037 737 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
W20260812 19:07:39.742560 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 214420us
I20260812 19:07:39.746008 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.746832 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.749632 29342 raft_consensus.cc:1226] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:39.676237 932 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.756363 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.724732 905 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.743134 686 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:39.746819 30679 raft_consensus.cc:1261] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.763032 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:39.764374 29337 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:39.754585 825 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:39.768529 1024 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:39.768676 713 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.762888 1000 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.743680 980 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.741999 924 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.764567 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.761518 959 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:39.774683 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.786319 29458 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:39.747164 904 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:39.789511 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 111577us
W20260812 19:07:39.790428 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 368014us
I20260812 19:07:39.791621 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:39.798342 29445 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:39.799769 29335 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:39.770462 666 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260812 19:07:39.802459 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:39.804967 29451 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:39.774770 825 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.817142 932 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:39.803807 29452 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:39.796339 959 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:39.825620 29569 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.822274 30686 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260812 19:07:39.825100 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.828867 29452 raft_consensus.cc:595] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.826545 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:39.832352 29456 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:39.823378 30685 raft_consensus.cc:1226] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:39.821820 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.835249 29572 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260812 19:07:39.835645 29572 pending_rounds.cc:76] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting all ops after (but not including) 4
I20260812 19:07:39.835845 29572 pending_rounds.cc:97] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260812 19:07:39.839169 29572 raft_consensus.cc:2866] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260812 19:07:39.840031 957 log.cc:924] Time spent T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 1.273s user 0.000s sys 0.007s
I20260812 19:07:39.825253 29574 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:39.841998 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.843328 29450 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:39.817725 1038 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:39.758945 997 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:39.846534 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:39.846840 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 51413.5us
I20260812 19:07:39.847954 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:39.850334 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:39.851275 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:39.847750 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:39.855475 29452 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:39.845201 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.855763 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:39.856191 29451 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:39.856989 29458 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:39.858083 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:39.841353 1039 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:39.839942 969 consensus_queue.cc:1046] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260812 19:07:39.862996 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:39.863207 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.879318 29450 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:39.882555 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:39.885696 29575 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:39.822363 948 consensus_queue.cc:1046] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260812 19:07:39.800843 1000 consensus_queue.cc:1046] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260812 19:07:39.819732 1033 consensus_queue.cc:1046] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260812 19:07:39.862414 905 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:39.836199 647 consensus_queue.cc:1046] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260812 19:07:39.894910 1034 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:39.936215 29451 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:39.932022 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:39.819252 686 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:39.861642 693 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260812 19:07:39.898590 1028 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260812 19:07:39.863153 898 consensus_queue.cc:1035] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.131s
I20260812 19:07:40.045017 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.047042 29337 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:39.985086 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 86013.9us
I20260812 19:07:40.046686 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:40.085371 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.079268 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:39.888662 737 ts_tablet_manager.cc:1428] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Time spent starting tablet: real 0.877s user 0.011s sys 0.000s
I20260812 19:07:39.882652 1011 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:40.026535 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1858) took 1400 ms. Trace:
I20260812 19:07:40.088622 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.092446 29459 rpcz_store.cc:276] 0812 19:07:38.625779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.625899 (+ 120us) service_pool.cc:227] Handling call
0812 19:07:38.626277 (+ 378us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:39.053946 (+427669us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:40.026268 (+972322us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54003f5054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:40.026467 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":797096704}
I20260812 19:07:40.093353 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.086766 29451 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:40.094164 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.094549 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.883638 917 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:39.932335 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:39.992556 536 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:39.932192 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:40.029189 989 log.cc:924] Time spent T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.129s user 0.000s sys 0.000s
I20260812 19:07:40.094549 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:39.888913 811 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:39.945081 1040 log.cc:924] Time spent T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.103s user 0.000s sys 0.000s
I20260812 19:07:40.040084 924 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:40.028911 925 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:40.099797 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.237257 29575 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:40.030205 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:40.110342 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.247639 29576 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:40.111011 29459 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:39.980834 29450 raft_consensus.cc:1226] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:40.086401 30676 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:40.085879 686 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:40.174240 29528 leader_election.cc:334] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.873s (SENT)
I20260812 19:07:40.077092 904 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:40.103942 737 ts_tablet_manager.cc:1460] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Tablet startup took 1100ms
I20260812 19:07:40.101395 642 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:40.112344 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:39.986145 669 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.221s
I20260812 19:07:40.109643 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.284461 29451 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260812 19:07:40.286219 29528 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:40.282110 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.287575 29282 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:40.293547 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 119366us
I20260812 19:07:40.048342 959 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.293390 29446 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:40.239156 30671 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:40.207672 1011 raft_consensus.cc:686] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:40.104728 771 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
W20260812 19:07:40.238075 912 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.516s user 0.010s sys 0.000s
I20260812 19:07:40.310527 29576 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:40.207096 825 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.312086 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.315685 1011 consensus_queue.cc:237] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.280413 30670 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:40.253381 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 761) took 1444 ms. Trace:
I20260812 19:07:40.331547 30623 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:40.349313 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.354777 29338 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:40.356189 29451 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:40.292307 998 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:40.359570 29459 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260812 19:07:40.362290 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 258880us
I20260812 19:07:40.301409 693 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:40.238734 536 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.251005 925 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.346156 825 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:40.399794 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.412361 29451 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:40.410737 29338 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8. Ops: []
W20260812 19:07:40.412974 29579 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 518) took 1850 ms (client timeout 2210 ms). Trace:
I20260812 19:07:40.248852 1023 consensus_queue.cc:1046] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
W20260812 19:07:40.413364 29579 rpcz_store.cc:269] 0812 19:07:38.562366 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.562487 (+ 121us) service_pool.cc:227] Handling call
0812 19:07:40.412826 (+1850339us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400134f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:40.412919 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":745438080}
I20260812 19:07:40.415143 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 637) took 1010 ms. Trace:
I20260812 19:07:40.415697 29335 raft_consensus.cc:1226] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.364127 929 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:40.417582 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:40.425206 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 90005.1us
I20260812 19:07:40.421813 29450 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
W20260812 19:07:40.427867 29528 leader_election.cc:334] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.699s (SENT)
I20260812 19:07:40.439062 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:40.439306 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:40.441238 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.442075 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.417667 29334 rpcz_store.cc:276] 0812 19:07:39.404716 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.405019 (+ 303us) service_pool.cc:227] Handling call
0812 19:07:39.418611 (+ 13592us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:39.424451 (+ 5840us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:39.424482 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:39.449584 (+ 25102us) log.cc:842] Serialized 26 byte log entry
0812 19:07:39.743109 (+293525us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:39.743167 (+ 58us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:39.743233 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:40.414915 (+671682us) raft_consensus.cc:1671] finished
0812 19:07:40.414937 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:40.415084 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5492,"raft.queue_time_us":34500,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":48171,"thread_start_us":237750,"threads_started":1,"wal-append.queue_time_us":324390}
I20260812 19:07:40.444042 29334 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
W20260812 19:07:40.410348 777 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260812 19:07:40.399411 1061 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:40.441581 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.453444 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.410557 1034 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
W20260812 19:07:40.421159 763 consensus_meta.cc:287] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 1.092s user 0.006s sys 0.007s
W20260812 19:07:40.420768 958 consensus_meta.cc:287] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.338s user 0.008s sys 0.009s
I20260812 19:07:40.369773 693 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.351634 30675 rpcz_store.cc:276] 0812 19:07:38.802954 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.807701 (+ 4747us) service_pool.cc:227] Handling call
0812 19:07:39.945330 (+1137629us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:40.075252 (+129922us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.088550 (+ 13298us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.095304 (+ 6754us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:40.103070 (+ 7766us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:40.109942 (+ 6872us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:40.184481 (+ 74539us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:40.247055 (+ 62574us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":14830848}
I20260812 19:07:40.445326 771 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:40.446429 1023 consensus_queue.cc:1225] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260812 19:07:40.371783 998 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.463178 1039 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:40.525166 1039 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.526268 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.526887 29569 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:40.479130 29332 raft_consensus.cc:1261] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:40.477402 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:40.487327 1062 log.cc:924] Time spent T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260812 19:07:40.433437 1066 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:40.447530 929 raft_consensus.cc:686] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:40.477773 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:40.464799 1051 log.cc:924] Time spent T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.478s user 0.000s sys 0.001s
I20260812 19:07:40.504535 771 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.487515 708 raft_consensus.cc:3032] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:40.490134 776 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.485916 1060 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:40.491729 1034 raft_consensus.cc:686] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:40.494019 978 consensus_queue.cc:1046] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260812 19:07:40.522969 29580 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 517) took 1972 ms (client timeout 1810 ms). Trace:
W20260812 19:07:40.623835 29580 rpcz_store.cc:269] 0812 19:07:38.550839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.552844 (+ 2005us) service_pool.cc:227] Handling call
0812 19:07:40.522754 (+1969910us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54000afd54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:40.522883 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":954097536}
I20260812 19:07:40.524286 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2119) took 1724 ms. Trace:
I20260812 19:07:40.624948 29339 rpcz_store.cc:276] 0812 19:07:38.799618 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.846078 (+ 46460us) service_pool.cc:227] Handling call
0812 19:07:38.875169 (+ 29091us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:39.262906 (+387737us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:40.524070 (+1261164us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400091ad4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:40.524230 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":872388352}
I20260812 19:07:40.465786 1061 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.524633 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2224) took 1384 ms. Trace:
W20260812 19:07:40.589735 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 161892us
I20260812 19:07:40.598573 1069 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:40.599154 29446 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.601100 30674 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:40.603047 30684 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:40.603053 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:40.621008 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.050s sys 0.086s
I20260812 19:07:40.626017 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:40.631633 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.558s user 0.050s sys 0.087s
I20260812 19:07:40.626953 29342 rpcz_store.cc:276] 0812 19:07:39.139909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.205060 (+ 65151us) service_pool.cc:227] Handling call
0812 19:07:39.205420 (+ 360us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:39.754525 (+549105us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:40.524414 (+769889us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54003d7554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:40.524577 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1251133824}
I20260812 19:07:40.627918 1078 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:40.499352 958 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:40.626132 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 273052us
I20260812 19:07:40.608690 708 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:40.610049 713 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:40.629930 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:40.631963 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 558271us
I20260812 19:07:40.622102 1060 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:40.635809 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:40.642364 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.643683 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.646363 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.648707 29450 raft_consensus.cc:1226] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.649533 29457 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:40.647923 1034 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.633436 925 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.655969 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.634258 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.610337 1066 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.654902 958 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:40.645079 536 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.674556 29450 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:40.671156 30681 raft_consensus.cc:1203] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Deduplicated request from leader. Original: 1.2->[3.3-3.3] Dedup: 3.3->[]
I20260812 19:07:40.682408 30675 raft_consensus.cc:2354] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 7.
I20260812 19:07:40.684008 1061 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.684505 30675 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:40.648466 713 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.686453 29459 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260812 19:07:40.686796 29459 pending_rounds.cc:76] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 5
I20260812 19:07:40.686997 29459 pending_rounds.cc:97] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260812 19:07:40.687203 29459 raft_consensus.cc:2866] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:40.651778 693 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:40.667260 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:40.698210 29569 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:07:40.288282 737 ts_tablet_manager.cc:1462] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Trace:
0812 19:07:33.134219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:33.136451 (+ 2232us) service_pool.cc:227] Handling call
0812 19:07:33.342103 (+205652us) ts_tablet_manager.cc:928] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Initiating tablet copy from peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295)
0812 19:07:33.855397 (+513294us) tablet_copy_client.cc:361] Tablet copy session begun
0812 19:07:35.134928 (+1279531us) tablet_metadata.cc:641] Metadata flushed
0812 19:07:35.135652 (+ 724us) tablet_copy_client.cc:490] Wrote new tablet metadata
0812 19:07:35.938219 (+802567us) tablet_copy_client.cc:411] Wrote new consensus metadata
0812 19:07:35.980614 (+ 42395us) tablet_replica.cc:205] Creating consensus instance
0812 19:07:35.981282 (+ 668us) ts_tablet_manager.cc:983] Replica registered.
0812 19:07:35.982601 (+ 1319us) inbound_call.cc:173] Queueing success response
0812 19:07:37.783075 (+1800474us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0812 19:07:39.018419 (+1235344us) ts_tablet_manager.cc:1429] Starting tablet replica
0812 19:07:39.126880 (+108461us) tablet_replica.cc:259] Starting instrumentation
0812 19:07:39.127460 (+ 580us) tablet_replica.cc:278] Starting consensus
Metrics: {"dns_us":249,"raft.queue_time_us":1084,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":3166,"tablet-copy.queue_time_us":74960,"tablet-download-pool.queue_time_us":18648,"tablet-download-pool.run_cpu_time_us":1473,"tablet-download-pool.run_wall_time_us":548586,"thread_start_us":86874,"threads_started":2}
I20260812 19:07:40.687655 925 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:40.655334 998 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:40.704948 30625 leader_election.cc:334] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.754s (SENT)
I20260812 19:07:40.685127 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:40.682646 30681 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:40.709378 30685 raft_consensus.cc:2379] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:40.719079 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:40.727097 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.748585 30625 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:40.687847 1039 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:40.646288 708 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.757056 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:40.699860 713 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.747555 29572 raft_consensus.cc:2362] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4. Re-sending same reply.
I20260812 19:07:40.759640 29575 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:40.721633 997 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:40.646317 1060 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.765902 29569 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:40.777531 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:40.717247 1038 consensus_meta.cc:287] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.854s user 0.008s sys 0.010s
I20260812 19:07:40.765408 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:07:40.771881 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 66856.8us
I20260812 19:07:40.785799 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.787169 29572 raft_consensus.cc:2362] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2. Re-sending same reply.
I20260812 19:07:40.645247 1069 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.792308 1066 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:40.793993 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.794638 29456 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:40.795779 1060 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:40.663617 929 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.764967 771 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:40.750115 642 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
W20260812 19:07:40.825567 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 121646us
I20260812 19:07:40.750629 925 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.826908 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.826972 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:40.827700 29446 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:40.831385 29450 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260812 19:07:40.803791 1038 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:40.813634 708 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:40.838594 29282 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:40.788865 1039 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:40.842006 29452 raft_consensus.cc:1226] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:40.844446 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:40.832293 945 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:40.813634 30670 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
W20260812 19:07:40.852007 30671 consensus_meta.cc:287] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.536s user 0.014s sys 0.014s
I20260812 19:07:40.655614 1078 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:40.871752 29450 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.861585 945 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:40.862023 30671 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:40.877924 30684 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:40.876709 686 consensus_meta.cc:287] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.587s user 0.011s sys 0.006s
I20260812 19:07:40.872480 30675 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.881283 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:40.882009 29446 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:40.876318 945 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:40.902890 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:40.905926 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 642) took 1348 ms. Trace:
I20260812 19:07:40.906387 29338 rpcz_store.cc:276] 0812 19:07:39.557617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.430834 (+873217us) service_pool.cc:227] Handling call
0812 19:07:40.462074 (+ 31240us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:40.467464 (+ 5390us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.467492 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:40.591280 (+123788us) log.cc:842] Serialized 26 byte log entry
0812 19:07:40.860282 (+269002us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:40.860317 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:40.860379 (+ 62us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:40.905701 (+ 45322us) raft_consensus.cc:1671] finished
0812 19:07:40.905725 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:40.905867 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":35215,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6930,"thread_start_us":245283,"threads_started":1,"wal-append.queue_time_us":272639}
I20260812 19:07:40.908048 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:40.908788 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:40.899366 945 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:40.913093 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 72809us
I20260812 19:07:40.915474 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:40.916740 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2331) took 1069 ms. Trace:
W20260812 19:07:40.911502 30676 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.590s user 0.006s sys 0.021s
I20260812 19:07:40.918658 29458 rpcz_store.cc:276] 0812 19:07:39.847426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.908562 (+1061136us) service_pool.cc:227] Handling call
0812 19:07:40.916675 (+ 8113us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:40.917852 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:40.920131 30625 leader_election.cc:334] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.182s (SENT)
I20260812 19:07:40.922078 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:40.923339 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:40.924964 29338 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:40.926259 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:40.929301 1084 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260812 19:07:40.929592 686 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:40.918514 713 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:07:40.932327 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:40.918987 925 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.949765 713 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:40.950374 713 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:40.951326 713 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.956826 29281 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:40.958158 1034 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:40.959005 1034 raft_consensus.cc:686] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:40.960175 1034 consensus_queue.cc:237] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:40.963406 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:40.964102 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:40.965766 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 45667.4us
I20260812 19:07:40.967201 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:40.919829 30676 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:40.966666 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:40.967844 29338 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:40.970928 29450 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:40.971902 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:40.921098 1078 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:40.978413 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:40.979846 29338 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260812 19:07:40.982504 29451 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:40.982893 849 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:40.983603 922 consensus_queue.cc:1046] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:40.985750 29338 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:40.987015 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:41.062716 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 73195.6us
I20260812 19:07:41.110360 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 648) took 1250 ms. Trace:
I20260812 19:07:40.972527 29572 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:41.112314 29342 rpcz_store.cc:276] 0812 19:07:39.860389 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.642636 (+782247us) service_pool.cc:227] Handling call
0812 19:07:40.642874 (+ 238us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.870469 (+227595us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.870500 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:40.870538 (+ 38us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:40.870566 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.110096 (+239530us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54003d7a54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:41.110150 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.110298 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":522997632}
W20260812 19:07:41.039538 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 52147.2us
I20260812 19:07:40.987226 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:41.063006 1011 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260812 19:07:41.111722 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2226) took 1574 ms. Trace:
I20260812 19:07:41.116454 29335 rpcz_store.cc:276] 0812 19:07:39.537405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.800392 (+262987us) service_pool.cc:227] Handling call
0812 19:07:40.415532 (+615140us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.431718 (+ 16186us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.111539 (+679821us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400356450. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:41.111667 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1414200192}
I20260812 19:07:41.134068 29281 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:41.134932 799 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:41.135324 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:41.136153 1087 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:41.136656 1087 raft_consensus.cc:686] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:41.138576 29456 raft_consensus.cc:3032] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:41.138927 29456 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:40.985625 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 712) took 1094 ms. Trace:
W20260812 19:07:41.140182 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 99683.9us
I20260812 19:07:41.140225 29281 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:41.142155 29335 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:41.143245 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:41.143996 29451 raft_consensus.cc:2354] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4.
I20260812 19:07:41.111835 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:41.159858 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.159091 29446 raft_consensus.cc:1226] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:41.042306 30670 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260812 19:07:41.163316 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.163502 706 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.070889 1024 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:41.034670 849 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.163558 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.166556 29456 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.041599 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:41.183207 29456 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:41.195933 29569 raft_consensus.cc:3032] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:41.196365 29569 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:41.137533 1087 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.147310 925 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:41.206709 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.206331 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.140270 30676 rpcz_store.cc:276] 0812 19:07:39.881122 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.883641 (+ 2519us) service_pool.cc:227] Handling call
0812 19:07:40.975769 (+1092128us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:41.144783 772 consensus_queue.cc:1046] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260812 19:07:41.244567 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.154366 642 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260812 19:07:41.248754 686 consensus_queue.cc:1046] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:41.268965 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:41.271804 1091 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:41.272333 1091 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:41.273833 1017 raft_consensus.cc:3032] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:41.274376 1017 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:41.274935 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.250392 29569 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.281323 29569 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:41.276016 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:41.282689 29458 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:41.184654 1024 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:41.228015 711 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:41.206183 693 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.306252 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:41.315147 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 151918us
I20260812 19:07:41.315948 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:41.318739 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 49859.1us
I20260812 19:07:41.313551 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.291393 772 consensus_queue.cc:1225] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260812 19:07:41.273630 1091 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.338161 1091 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:41.194339 706 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.337379 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.350008 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:41.291925 925 raft_consensus.cc:686] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:41.203593 849 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:41.378075 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 58161.6us
W20260812 19:07:41.346375 1019 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.262s user 0.003s sys 0.015s
I20260812 19:07:41.388136 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.399118 29399 leader_election.cc:398] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 7.
I20260812 19:07:41.381172 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.324731 711 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:41.324983 642 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:41.389717 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.401624 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:41.403679 29451 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:41.347203 669 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:41.413259 29458 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:41.414439 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:41.415969 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.416692 29569 raft_consensus.cc:1261] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:41.419076 29456 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260812 19:07:41.402943 1019 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.323s user 0.004s sys 0.017s
I20260812 19:07:41.391744 1011 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.324540 693 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.325214 1017 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.400058 924 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:41.410326 1060 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.614s user 0.014s sys 0.007s
I20260812 19:07:41.403003 711 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:41.402251 630 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:41.419453 1039 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:41.422513 30671 raft_consensus.cc:3032] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
W20260812 19:07:41.424880 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 196078us
I20260812 19:07:41.428673 1011 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:41.424698 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 107685us
I20260812 19:07:41.437760 711 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:41.430450 1126 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:41.422394 669 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:41.440205 708 consensus_meta.cc:287] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.596s user 0.008s sys 0.024s
I20260812 19:07:41.454720 708 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:07:41.440052 30671 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:41.456753 1052 log.cc:924] Time spent T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 1.159s user 0.000s sys 0.002s
I20260812 19:07:41.456667 30671 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.434296 925 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:41.451068 29341 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2217) took 2856 ms (client timeout 1666 ms). Trace:
I20260812 19:07:41.454851 29576 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260812 19:07:41.442721 945 consensus_meta.cc:287] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.517s user 0.010s sys 0.008s
I20260812 19:07:41.436714 924 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.453512 825 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:41.457943 30671 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:41.459797 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 684740us
I20260812 19:07:41.458307 1076 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:41.466257 1076 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:41.459563 1121 log.cc:924] Time spent T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260812 19:07:41.462822 945 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.464426 1126 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:07:41.458998 29528 leader_election.cc:334] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.811s (SENT)
I20260812 19:07:41.460191 1036 raft_consensus.cc:3032] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:41.470549 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.439988 1017 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:41.461472 29341 rpcz_store.cc:269] 0812 19:07:38.594582 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.602232 (+ 7650us) service_pool.cc:227] Handling call
0812 19:07:41.450826 (+2848594us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400012cd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:41.450978 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2099657472}
I20260812 19:07:41.461886 29527 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:41.480172 29572 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:41.460027 1066 consensus_queue.cc:1046] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260812 19:07:41.480043 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 542) took 1174 ms. Trace:
I20260812 19:07:41.480906 29530 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:41.481426 645 consensus_queue.cc:1046] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
W20260812 19:07:41.496510 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 91152us
I20260812 19:07:41.411522 1024 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.499303 29331 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
W20260812 19:07:41.492396 29282 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.885s (SENT) [suppressed 40 similar messages]
I20260812 19:07:41.494350 29578 rpcz_store.cc:276] 0812 19:07:40.305749 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.305884 (+ 135us) service_pool.cc:227] Handling call
0812 19:07:40.316198 (+ 10314us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.261089 (+944891us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18002c1520. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c69a9d82c 00007f9c66ed49dc 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66eef155 00007f9c69e063d0 00007f9c69e06f6b 00007f9c69e1178b
0812 19:07:41.261249 (+ 160us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.261272 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:41.261717 (+ 445us) log.cc:842] Serialized 26 byte log entry
0812 19:07:41.425393 (+163676us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:41.425546 (+ 153us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.432292 (+ 6746us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:41.452743 (+ 20451us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18002c1520. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66e694ca 00007f9c7053ef99 00007f9c6dcb6bcf 00007f9c66ef27b6 00007f9c66ef5b11 00007f9c66ef5ac9 00007f9c66ef5a59 00007f9c66ef5a21 00007f9c66ef4d1d
0812 19:07:41.479780 (+ 27037us) raft_consensus.cc:1671] finished
0812 19:07:41.479804 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.479981 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1754,"raft.queue_time_us":105953,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":16053,"spinlock_wait_cycles":12837376,"thread_start_us":302188,"threads_started":2,"wal-append.queue_time_us":198299}
I20260812 19:07:41.501911 29528 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.504063 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:41.508276 29341 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:41.509444 29341 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:41.511770 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:41.485297 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 652) took 1062 ms. Trace:
I20260812 19:07:41.513037 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:41.513175 30671 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:41.516638 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.519485 29456 raft_consensus.cc:1261] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260812 19:07:41.470803 1036 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:41.528422 29528 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.496486 664 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:41.441275 1039 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.499634 669 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:41.470784 945 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.536489 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:41.538173 30625 leader_election.cc:398] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4.
I20260812 19:07:41.548832 30625 leader_election.cc:401] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260812 19:07:41.479506 642 consensus_queue.cc:1046] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260812 19:07:41.529140 1018 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260812 19:07:41.464360 825 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.531479 706 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.531071 1024 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:41.530119 1093 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:41.556619 30623 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:41.538214 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:41.534809 708 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:41.526552 771 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:41.547643 664 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:41.563316 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 129283us
W20260812 19:07:41.546051 997 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.769s user 0.010s sys 0.005s
I20260812 19:07:41.526463 1080 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:41.570838 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 653) took 1047 ms. Trace:
I20260812 19:07:41.567628 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:41.512818 29338 rpcz_store.cc:276] 0812 19:07:40.422686 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.039469 (+616783us) service_pool.cc:227] Handling call
0812 19:07:41.039847 (+ 378us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.040050 (+ 203us) raft_consensus.cc:1520] Early marking committed up to index 5
0812 19:07:41.402504 (+362454us) log.cc:842] Serialized 14 byte log entry
0812 19:07:41.484933 (+ 82429us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:41.484996 (+ 63us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:41.485046 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.485118 (+ 72us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.485234 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":396112,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":1650,"raft.queue_time_us":86,"spinlock_wait_cycles":6574848,"thread_start_us":443472,"threads_started":2}
W20260812 19:07:41.567586 981 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.521s user 0.008s sys 0.012s
I20260812 19:07:41.572302 708 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:41.568832 1093 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:41.579960 29451 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:41.545926 1126 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.612023 1066 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:41.595505 29342 rpcz_store.cc:276] 0812 19:07:40.523456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.114002 (+590546us) service_pool.cc:227] Handling call
0812 19:07:41.114302 (+ 300us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.114604 (+ 302us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:41.416925 (+302321us) log.cc:842] Serialized 14 byte log entry
0812 19:07:41.570457 (+153532us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:41.570507 (+ 50us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:41.570554 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.570616 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.570721 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":344981,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":21964,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":436708,"threads_started":2}
I20260812 19:07:41.572736 1011 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:41.604497 981 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.557s user 0.008s sys 0.012s
I20260812 19:07:41.612986 1066 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.578547 849 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:41.577543 771 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:41.578150 1076 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.606803 811 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:41.667068 29342 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:41.670423 693 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:41.606208 959 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.673213 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:41.673908 1124 log.cc:924] Time spent T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.234s user 0.000s sys 0.001s
I20260812 19:07:41.606181 987 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:41.667995 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.606010 536 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:41.655364 708 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:41.686370 29338 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:41.677729 771 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:41.669962 1146 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.665033 924 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.699048 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.723618 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.676534 1093 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.605403 1080 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.725409 30625 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.685139 1076 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:41.706441 29342 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:41.682863 811 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:41.594733 664 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.727703 30625 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.705094 849 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:41.744736 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2136) took 1001 ms. Trace:
W20260812 19:07:41.723512 799 consensus_meta.cc:287] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.586s user 0.010s sys 0.004s
I20260812 19:07:41.746424 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.751910 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:41.748416 29342 rpcz_store.cc:276] 0812 19:07:40.743609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.684013 (+940404us) service_pool.cc:227] Handling call
0812 19:07:41.686113 (+ 2100us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.744098 (+ 57985us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.744677 (+ 579us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:41.662026 1036 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.759261 1036 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:41.737735 778 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:41.734030 709 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:41.733605 669 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:41.744364 1093 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:41.743451 811 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:41.695705 959 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:41.756196 1114 log.cc:924] Time spent T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.215s user 0.001s sys 0.001s
I20260812 19:07:41.760083 799 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:41.753290 1083 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:41.779400 1122 log.cc:924] Time spent T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.190s user 0.000s sys 0.001s
I20260812 19:07:41.781986 1136 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.786070 29341 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:41.745441 1076 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.772224 778 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:41.768582 1039 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:41.774796 709 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260812 19:07:41.801424 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 273071us
W20260812 19:07:41.798523 1138 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.220s user 0.001s sys 0.000s
I20260812 19:07:41.791400 1070 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.791240 825 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:41.744987 1146 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.881868 1076 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.883981 30679 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:41.884936 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 781) took 1275 ms. Trace:
I20260812 19:07:41.863107 645 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:41.806454 709 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:41.896121 29528 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:41.897397 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.898777 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:41.794806 1083 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:41.900472 777 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260812 19:07:41.901134 1137 log.cc:924] Time spent T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.170s user 0.000s sys 0.001s
W20260812 19:07:41.886868 1091 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.548s user 0.010s sys 0.002s
I20260812 19:07:41.876545 642 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.783063 1129 consensus_queue.cc:1046] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.310s
I20260812 19:07:41.921481 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 649) took 2032 ms. Trace:
I20260812 19:07:41.921916 29339 rpcz_store.cc:276] 0812 19:07:39.889379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.650408 (+761029us) service_pool.cc:227] Handling call
0812 19:07:40.660955 (+ 10547us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.688749 (+ 27794us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.688781 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:40.688820 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:40.688851 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.921202 (+1232351us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b5400356954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:41.921265 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.921422 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":63506688}
I20260812 19:07:41.904106 709 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:41.922955 1150 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.151s user 0.000s sys 0.001s
I20260812 19:07:41.924633 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2322) took 2132 ms. Trace:
I20260812 19:07:41.925031 29452 rpcz_store.cc:276] 0812 19:07:39.791756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.026588 (+234832us) service_pool.cc:227] Handling call
0812 19:07:40.068986 (+ 42398us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.843591 (+774605us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.924389 (+1080798us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b5400212354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:41.924568 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1775888256}
I20260812 19:07:41.905333 777 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:41.910971 1080 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.914359 825 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:41.915518 1064 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:41.918521 642 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.919142 998 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:41.889341 1066 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.927379 29458 raft_consensus.cc:2396] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:07:41.886795 645 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.885540 30679 rpcz_store.cc:276] 0812 19:07:40.609208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.611578 (+ 2370us) service_pool.cc:227] Handling call
0812 19:07:41.883168 (+1271590us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66ebabce 000000000034aea9 00007f9c6ecdaa30 00007f9c6eda07b4 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:41.883435 (+ 267us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.884598 (+ 1163us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.884875 (+ 277us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4443008}
I20260812 19:07:41.905835 1083 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:41.940054 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:41.927366 1076 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:41.930263 777 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:41.937599 664 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.942233 1038 raft_consensus.cc:3032] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:41.907357 1070 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:41.862398 1136 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.951205 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 645) took 2241 ms. Trace:
I20260812 19:07:41.956521 29334 rpcz_store.cc:276] 0812 19:07:39.709367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.454770 (+745403us) service_pool.cc:227] Handling call
0812 19:07:40.455059 (+ 289us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.644724 (+189665us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.644757 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:40.644798 (+ 41us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:40.644836 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.950959 (+1306123us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400231ed4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:41.951014 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:41.951141 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":435752192}
I20260812 19:07:41.936784 711 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:41.953207 980 raft_consensus.cc:3032] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:41.954372 30676 pending_rounds.cc:76] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Aborting all ops after (but not including) 6
I20260812 19:07:41.938346 1064 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:41.956440 1023 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:41.949750 1038 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:41.968297 30676 pending_rounds.cc:97] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260812 19:07:41.943506 1066 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260812 19:07:41.977809 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 485426us
I20260812 19:07:41.961737 980 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:41.969726 1146 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:41.995813 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 439261us
I20260812 19:07:41.999070 30623 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:41.974306 1023 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:41.977994 959 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:41.988885 1076 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:41.981894 30676 raft_consensus.cc:2866] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:42.009940 29339 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:41.986939 825 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:42.007807 1129 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:42.012497 533 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260812 19:07:42.018617 1129 raft_consensus.cc:686] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:42.005802 1023 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:42.029340 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.030047 29567 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:42.029719 711 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.035786 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.057451 29576 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260812 19:07:42.079706 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.096935 30623 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:42.109416 1081 raft_consensus.cc:3032] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:42.110119 1081 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:42.025138 771 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.110276 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.111121 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.111820 29580 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:42.116463 29580 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:42.077814 642 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.120239 669 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:42.096578 645 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:42.122591 771 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.021486 825 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:42.014868 1076 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.132884 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.038245 998 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260812 19:07:42.014451 1064 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:42.020362 1129 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.148155 533 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:42.116377 1081 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.025662 1038 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.144572 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.124701 936 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:42.051031 959 consensus_queue.cc:1046] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260812 19:07:42.169250 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.171537 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2227) took 2633 ms. Trace:
I20260812 19:07:42.171940 29337 rpcz_store.cc:276] 0812 19:07:39.537717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.839048 (+301331us) service_pool.cc:227] Handling call
0812 19:07:40.046857 (+207809us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.085289 (+ 38432us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.171307 (+2086018us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54002319d0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:42.171441 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":477227904}
I20260812 19:07:42.173050 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.177028 1038 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:42.036444 980 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:42.178869 980 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.178962 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2116) took 3380 ms. Trace:
I20260812 19:07:42.180601 29331 rpcz_store.cc:276] 0812 19:07:38.798060 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:38.798164 (+ 104us) service_pool.cc:227] Handling call
0812 19:07:41.451305 (+2653141us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.507313 (+ 56008us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.948357 (+441044us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b5400012cd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.178751 (+230394us) spinlock_profiling.cc:243] Waited 2.65 s on lock 0x7b5400012cd0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:42.178890 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6210083456}
I20260812 19:07:42.206404 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 543) took 1900 ms. Trace:
I20260812 19:07:42.210222 959 consensus_queue.cc:1225] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260812 19:07:42.210915 959 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:42.211532 959 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:42.212899 959 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:42.181129 1064 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.184082 936 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:42.295466 959 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.298969 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2137) took 1048 ms. Trace:
I20260812 19:07:42.352363 29339 rpcz_store.cc:276] 0812 19:07:41.250769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.923039 (+672270us) service_pool.cc:227] Handling call
0812 19:07:41.923629 (+ 590us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.021207 (+ 97578us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.298675 (+277468us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b5400091ad4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.298898 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":184782720}
I20260812 19:07:42.365717 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 656) took 1386 ms. Trace:
I20260812 19:07:42.386088 29338 rpcz_store.cc:276] 0812 19:07:40.979752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.702905 (+723153us) service_pool.cc:227] Handling call
0812 19:07:41.744279 (+ 41374us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.744471 (+ 192us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:41.762472 (+ 18001us) log.cc:842] Serialized 14 byte log entry
0812 19:07:41.765538 (+ 3066us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:41.765603 (+ 65us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:41.765646 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:41.765710 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:42.365456 (+599746us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b54003d7050. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:42.365647 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2805,"prepare.queue_time_us":20498,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":3250,"spinlock_wait_cycles":27908608}
I20260812 19:07:42.414346 948 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:42.415024 948 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.446434 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.447000 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2345) took 2343 ms. Trace:
I20260812 19:07:42.266948 998 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:42.447374 29459 rpcz_store.cc:276] 0812 19:07:40.103182 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.368834 (+265652us) service_pool.cc:227] Handling call
0812 19:07:40.411011 (+ 42177us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.687739 (+276728us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.446771 (+1759032us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540038e854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.446930 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":632671616}
I20260812 19:07:42.455083 30675 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:42.448926 1024 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:42.456137 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 725) took 1487 ms. Trace:
I20260812 19:07:42.456058 1024 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.456698 30675 rpcz_store.cc:276] 0812 19:07:40.968298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.970000 (+ 1702us) service_pool.cc:227] Handling call
0812 19:07:42.449342 (+1479342us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:42.454380 (+ 5038us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.455710 (+ 1330us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.456049 (+ 339us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12983168}
W20260812 19:07:42.181988 917 consensus_queue.cc:1159] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" responder_term: 4 status { last_received { term: 4 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 4 index: 7 } }
I20260812 19:07:42.351758 936 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:42.464883 29453 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
W20260812 19:07:42.479244 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.900s user 0.007s sys 0.018s
W20260812 19:07:42.479794 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.901s user 0.007s sys 0.019s
I20260812 19:07:42.464582 29332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 636) took 3158 ms. Trace:
W20260812 19:07:42.492022 29457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39326 (request call id 1868) took 2916 ms (client timeout 1885 ms). Trace:
W20260812 19:07:42.492614 29457 rpcz_store.cc:269] 0812 19:07:39.548589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.548765 (+ 176us) service_pool.cc:227] Handling call
0812 19:07:42.464520 (+2915755us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003f4154. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:42.464711 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2504902528}
I20260812 19:07:42.494361 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.489812 29332 rpcz_store.cc:276] 0812 19:07:39.306152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.306854 (+ 702us) service_pool.cc:227] Handling call
0812 19:07:39.311030 (+ 4176us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:40.535332 (+1224302us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.464313 (+1928981us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540011cbd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.464508 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2685542656}
I20260812 19:07:42.208058 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.498051 29337 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:42.451603 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:42.205127 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.451347 1000 raft_consensus.cc:3032] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:42.519754 1136 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:42.522224 1070 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:42.524734 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 658) took 1248 ms. Trace:
I20260812 19:07:42.525115 29339 rpcz_store.cc:276] 0812 19:07:41.275906 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.458335 (+1182429us) service_pool.cc:227] Handling call
0812 19:07:42.524182 (+ 65847us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.524352 (+ 170us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.524376 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:42.524431 (+ 55us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:42.524463 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.524569 (+ 106us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:42.524675 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:42.206833 29577 rpcz_store.cc:276] 0812 19:07:40.306065 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.306170 (+ 105us) service_pool.cc:227] Handling call
0812 19:07:40.315457 (+ 9287us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:40.423173 (+107716us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:40.423204 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:40.424868 (+ 1664us) log.cc:842] Serialized 26 byte log entry
0812 19:07:40.708016 (+283148us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:40.708051 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.205988 (+1497937us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400156654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.206039 (+ 51us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:42.206083 (+ 44us) raft_consensus.cc:1671] finished
0812 19:07:42.206096 (+ 13us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:42.206343 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1031387,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"spinlock_wait_cycles":246813568,"thread_start_us":261788,"threads_started":1,"wal-append.queue_time_us":418082,"wal-append.run_cpu_time_us":2886,"wal-append.run_wall_time_us":1194238}
I20260812 19:07:42.538936 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:37750 (request call id 867) took 3122 ms. Trace:
I20260812 19:07:42.546700 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:42.539460 29331 rpcz_store.cc:276] 0812 19:07:39.416656 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.527858 (+3111202us) service_pool.cc:227] Handling call
0812 19:07:42.538867 (+ 11009us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:42.549459 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.550159 29332 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:42.556543 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.599757 29339 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:42.524246 726 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.513571 1000 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:42.459339 998 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.574633 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.575485 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.609208 29577 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:42.583472 825 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:42.583755 1024 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.544137 1155 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:42.623036 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 59701.8us
I20260812 19:07:42.642103 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:42.555735 763 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
W20260812 19:07:42.676611 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 87575.1us
W20260812 19:07:42.607404 1165 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.090s user 0.001s sys 0.000s
I20260812 19:07:42.690169 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:42.690873 29577 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:42.612120 702 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:42.694849 811 consensus_meta.cc:287] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.911s user 0.005s sys 0.014s
I20260812 19:07:42.694686 702 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.695662 1069 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.696307 776 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:42.695290 811 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:42.699424 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 57442.9us
I20260812 19:07:42.612085 948 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.711361 825 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:42.717339 30623 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:42.719852 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:42.720623 29577 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.716764 825 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:42.639468 1070 consensus_queue.cc:1046] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260812 19:07:42.723353 702 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:42.723372 825 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:07:42.725003 1017 consensus_meta.cc:287] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.226s user 0.015s sys 0.012s
I20260812 19:07:42.689148 1076 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.727865 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.732034 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:42.723776 1070 consensus_queue.cc:1225] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260812 19:07:42.725970 745 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:42.736910 1152 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:42.737928 1152 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:42.738593 1152 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:42.739344 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.743817 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.717849 763 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:42.725700 1017 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:42.743849 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.762387 1031 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:42.743319 29445 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.763s user 0.008s sys 0.011s
I20260812 19:07:42.745666 745 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:42.791009 777 consensus_meta.cc:287] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.837s user 0.007s sys 0.014s
I20260812 19:07:42.798241 29445 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:42.825660 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:42.826699 29331 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
W20260812 19:07:42.777438 1036 consensus_meta.cc:287] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.018s user 0.015s sys 0.019s
I20260812 19:07:42.776029 1017 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:42.834446 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2135) took 2111 ms. Trace:
I20260812 19:07:42.835772 29341 rpcz_store.cc:276] 0812 19:07:40.722878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.516188 (+793310us) service_pool.cc:227] Handling call
0812 19:07:41.516451 (+ 263us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.794943 (+278492us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.834191 (+1039248us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540011dad4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:42.834384 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":619579136}
I20260812 19:07:42.833760 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:42.843431 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.186s user 0.050s sys 0.077s
W20260812 19:07:42.844782 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.239s user 0.019s sys 0.031s
W20260812 19:07:42.845129 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.239s user 0.019s sys 0.031s
W20260812 19:07:42.845481 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.188s user 0.050s sys 0.077s
W20260812 19:07:42.802623 987 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.101s user 0.005s sys 0.027s
W20260812 19:07:42.831501 708 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.109s user 0.018s sys 0.006s
W20260812 19:07:42.863721 29400 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.206s user 0.024s sys 0.080s
W20260812 19:07:42.864073 29400 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.207s user 0.024s sys 0.080s
W20260812 19:07:42.863481 29342 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s user 0.009s sys 0.038s
W20260812 19:07:42.983860 29342 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.179s user 0.009s sys 0.038s
I20260812 19:07:42.687552 1155 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.986128 1155 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:42.847010 1093 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.073s user 0.005s sys 0.022s
I20260812 19:07:42.831595 929 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:42.915933 29402 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.223s user 0.019s sys 0.029s
W20260812 19:07:42.987604 29402 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.302s user 0.019s sys 0.029s
I20260812 19:07:42.789729 763 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:42.988353 1070 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:42.845249 1017 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:42.862684 1076 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:42.863515 987 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4.
I20260812 19:07:42.831959 777 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:42.870684 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.110s user 0.009s sys 0.026s
W20260812 19:07:42.884004 30684 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.301s user 0.027s sys 0.022s
W20260812 19:07:42.884424 928 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.147s user 0.008s sys 0.045s
I20260812 19:07:42.900985 998 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:42.905656 1163 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:42.912233 29456 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.345s user 0.022s sys 0.040s
I20260812 19:07:42.988937 1070 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:42.736801 1000 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.790831 1031 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:42.869668 708 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:43.089228 29456 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.522s user 0.023s sys 0.041s
W20260812 19:07:43.089066 30684 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.507s user 0.032s sys 0.022s
W20260812 19:07:42.917707 29579 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s user 0.013s sys 0.031s
W20260812 19:07:42.950080 29284 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.291s user 0.026s sys 0.055s
W20260812 19:07:43.089015 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.329s user 0.014s sys 0.030s
W20260812 19:07:43.097174 29284 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.438s user 0.026s sys 0.055s
W20260812 19:07:43.098966 29579 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s user 0.015s sys 0.036s
W20260812 19:07:43.107013 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 1.35506e+06us
I20260812 19:07:43.110747 30686 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:43.112694 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:43.095037 928 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.358s user 0.008s sys 0.049s
W20260812 19:07:43.091246 925 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.316s user 0.016s sys 0.044s
I20260812 19:07:43.114845 30625 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:43.119832 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:43.007258 1093 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:42.989434 763 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:42.918308 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.078074 1154 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.130024 1154 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.131997 1154 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.132469 1154 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.132867 1154 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.022276 1162 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.123040 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:43.118458 29574 raft_consensus.cc:2396] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260812 19:07:43.003276 959 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.000597 1036 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:43.127193 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.064337 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:43.124354 925 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.349s user 0.017s sys 0.045s
I20260812 19:07:43.101836 849 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:43.092202 811 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.090910 1024 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:43.134675 709 consensus_meta.cc:287] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.212s user 0.023s sys 0.001s
W20260812 19:07:43.153699 29578 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.169s user 0.026s sys 0.018s
I20260812 19:07:43.101192 1160 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.089023 1076 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.129081 1093 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.031662 1060 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260812 19:07:43.101797 1163 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.064419 1007 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.257s
I20260812 19:07:42.987210 929 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.155823 29335 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:43.135505 29577 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:43.138327 533 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260812 19:07:43.157275 29334 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.177s user 0.019s sys 0.014s
W20260812 19:07:43.159632 29578 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.175s user 0.026s sys 0.018s
I20260812 19:07:43.121431 686 consensus_queue.cc:1035] T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260812 19:07:43.157590 1166 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.143922 998 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.125173 934 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260812 19:07:43.157966 30676 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.924s user 0.013s sys 0.020s
W20260812 19:07:43.212204 30676 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s user 0.014s sys 0.020s
I20260812 19:07:43.138898 987 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260812 19:07:43.150523 1069 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.213337 1069 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.215281 1069 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.154415 772 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:43.162569 1076 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:43.221467 1064 consensus_meta.cc:287] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.919s user 0.015s sys 0.002s
I20260812 19:07:43.181107 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.222230 29577 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:43.146526 30686 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 1.
I20260812 19:07:43.226912 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.228125 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.228684 30686 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:43.229552 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:43.181661 29334 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.202s user 0.019s sys 0.015s
I20260812 19:07:43.232028 929 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.144338 1162 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.149057 1036 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:43.161729 1129 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.015s sys 0.014s
I20260812 19:07:43.234014 1093 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:43.234535 1129 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s user 0.015s sys 0.015s
I20260812 19:07:43.153121 959 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:43.236349 1036 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.237176 1036 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.237241 959 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.155715 1024 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.245757 959 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.270814 959 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.273900 959 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.274802 959 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:43.275461 29458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 648) took 2137 ms (client timeout 2691 ms). Trace:
I20260812 19:07:43.275583 959 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:43.276314 825 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.551s user 0.008s sys 0.000s
I20260812 19:07:43.248112 1076 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.281545 1076 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.242837 29577 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:43.154794 709 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.281912 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 788) took 2581 ms. Trace:
I20260812 19:07:43.282346 30670 rpcz_store.cc:276] 0812 19:07:40.700463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.872891 (+172428us) service_pool.cc:227] Handling call
0812 19:07:40.973151 (+100260us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.191382 (+218231us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.281701 (+2090319us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b540070d850. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:43.281846 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":209376128}
I20260812 19:07:43.284400 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.251020 1163 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.254130 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 657) took 2269 ms. Trace:
I20260812 19:07:43.288264 29342 rpcz_store.cc:276] 0812 19:07:40.984908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.752850 (+767942us) service_pool.cc:227] Handling call
0812 19:07:41.753347 (+ 497us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.766050 (+ 12703us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.766095 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:41.766548 (+ 453us) log.cc:842] Serialized 26 byte log entry
0812 19:07:42.984281 (+1217733us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:42.984321 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:42.984408 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.253852 (+269444us) raft_consensus.cc:1671] finished
0812 19:07:43.253879 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.254046 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8862,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":4462,"thread_start_us":1179370,"threads_started":1,"wal-append.queue_time_us":1408447}
I20260812 19:07:43.283533 709 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.259714 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 772) took 1757 ms. Trace:
I20260812 19:07:43.261085 1060 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:43.262744 29445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2335) took 3278 ms (client timeout 1941 ms). Trace:
W20260812 19:07:43.278316 726 consensus_meta.cc:287] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.666s user 0.009s sys 0.000s
I20260812 19:07:43.244817 29567 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:43.277582 959 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.162488 1160 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.239632 1154 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.237861 1036 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:43.283759 776 consensus_meta.cc:287] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.587s user 0.006s sys 0.004s
I20260812 19:07:43.289760 30684 rpcz_store.cc:276] 0812 19:07:41.502137 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.502260 (+ 123us) service_pool.cc:227] Handling call
0812 19:07:41.502680 (+ 420us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.553591 (+ 50911us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.553618 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:41.563798 (+ 10180us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.096227 (+1532429us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:43.096268 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.096340 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.259511 (+163171us) raft_consensus.cc:1671] finished
0812 19:07:43.259531 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.259654 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":35255,"raft.queue_time_us":31341,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":6553,"thread_start_us":1513636,"threads_started":1,"wal-append.queue_time_us":1594495}
I20260812 19:07:43.357034 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:43.293588 29445 rpcz_store.cc:269] 0812 19:07:39.984019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.094228 (+110209us) service_pool.cc:227] Handling call
0812 19:07:43.262540 (+3168312us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540006ccd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:43.262647 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4331460608}
I20260812 19:07:43.357807 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.271867 1024 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.295063 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:43.297374 29337 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s user 0.011s sys 0.040s
W20260812 19:07:43.361205 29337 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s user 0.011s sys 0.040s
W20260812 19:07:43.295619 1152 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.556s user 0.003s sys 0.006s
I20260812 19:07:43.289531 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2088) took 1245 ms. Trace:
W20260812 19:07:43.275887 29458 rpcz_store.cc:269] 0812 19:07:41.138031 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.415603 (+277572us) service_pool.cc:227] Handling call
0812 19:07:43.275260 (+1859657us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54003bcc54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:43.275401 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1173905792}
I20260812 19:07:43.283671 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.280769 29176 catalog_manager.cc:5495] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 5 to 6, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 6 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:43.358549 29577 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:43.235023 1093 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
W20260812 19:07:43.301715 1087 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.596s user 0.022s sys 0.043s
I20260812 19:07:43.276695 825 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.367875 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:43.387794 29580 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.259s user 0.083s sys 0.080s
I20260812 19:07:43.211094 998 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:43.363356 1038 consensus_meta.cc:287] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.180s user 0.009s sys 0.011s
I20260812 19:07:43.385406 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.307806 912 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:43.388129 29580 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.260s user 0.083s sys 0.081s
I20260812 19:07:43.207353 1166 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.382999 29176 catalog_manager.cc:5495] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: config changed from index -1 to 2, NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68) added, no longer has a pending config: VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65), VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66), VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67), NON_VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } } }
I20260812 19:07:43.395815 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:43.354418 1031 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.356674 708 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.357199 777 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:43.415531 29576 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s user 0.005s sys 0.069s
I20260812 19:07:43.364516 959 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:43.386281 1087 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.680s user 0.022s sys 0.043s
I20260812 19:07:43.392422 1136 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:43.397629 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.299410 1060 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.386006 1033 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.369644 1152 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.404075 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:43.365511 29574 rpcz_store.cc:276] 0812 19:07:42.044152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.051916 (+ 7764us) service_pool.cc:227] Handling call
0812 19:07:43.289316 (+1237400us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54000af854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:43.289472 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1986942848}
W20260812 19:07:43.475421 29282 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.011s sys 0.032s
W20260812 19:07:43.479507 29282 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.510s user 0.011s sys 0.033s
I20260812 19:07:43.490872 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:43.491585 29341 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:43.493600 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:43.501025 29342 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:43.476670 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 570) took 1324 ms. Trace:
I20260812 19:07:43.416639 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 731) took 2087 ms. Trace:
I20260812 19:07:43.407168 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1900) took 2162 ms. Trace:
I20260812 19:07:43.518360 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:43.518464 29456 rpcz_store.cc:276] 0812 19:07:41.244521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.520343 (+275822us) service_pool.cc:227] Handling call
0812 19:07:41.520663 (+ 320us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.521620 (+ 957us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.521652 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:41.536910 (+ 15258us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.106731 (+1569821us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:43.106773 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.106843 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.406965 (+300122us) raft_consensus.cc:1671] finished
0812 19:07:43.406986 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.407109 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":10680,"thread_start_us":1528408,"threads_started":1,"wal-append.queue_time_us":1828468}
I20260812 19:07:43.407335 1166 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.369596 1174 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.527007 1174 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:43.527474 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.528261 29574 raft_consensus.cc:2379] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:43.529103 1174 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:43.416085 29576 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.846s user 0.005s sys 0.069s
I20260812 19:07:43.418217 29176 catalog_manager.cc:5495] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:43.529122 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.419152 30685 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:43.394490 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.530582 1163 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.449810 29338 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:43.400393 912 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:43.467469 1181 log.cc:924] Time spent T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260812 19:07:43.531226 1163 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.531620 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.393677 726 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:43.399919 1070 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.519111 29331 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:43.533206 1163 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:43.533139 726 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.535346 726 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:43.530544 29176 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 4, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:07:43.537412 1182 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.089s user 0.000s sys 0.000s
I20260812 19:07:43.504199 29569 rpcz_store.cc:276] 0812 19:07:42.152679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.496676 (+343997us) service_pool.cc:227] Handling call
0812 19:07:42.496991 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.685406 (+188415us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.685441 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:42.685482 (+ 41us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:42.685519 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.476391 (+790872us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b540009ea54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:43.476453 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.476605 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":432782208}
I20260812 19:07:43.538843 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2139) took 2272 ms. Trace:
W20260812 19:07:43.539906 1178 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.071s user 0.000s sys 0.000s
I20260812 19:07:43.532864 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 546) took 2815 ms. Trace:
I20260812 19:07:43.539121 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.532859 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.544049 29577 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:43.545207 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:43.539206 29334 rpcz_store.cc:276] 0812 19:07:41.266872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.961177 (+694305us) service_pool.cc:227] Handling call
0812 19:07:41.970046 (+ 8869us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.973714 (+ 3668us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.973742 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:41.979521 (+ 5779us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.232316 (+1252795us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:43.232357 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.232421 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.538610 (+306189us) raft_consensus.cc:1671] finished
0812 19:07:43.538634 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.538782 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":113,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3388,"thread_start_us":1252446,"threads_started":1,"wal-append.queue_time_us":1412811}
W20260812 19:07:43.405121 1083 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.450s user 0.015s sys 0.015s
I20260812 19:07:43.537644 29176 catalog_manager.cc:5495] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:07:43.425606 1136 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.540823 29579 rpcz_store.cc:276] 0812 19:07:40.717730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.717871 (+ 141us) service_pool.cc:227] Handling call
0812 19:07:40.718354 (+ 483us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:41.462799 (+744445us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:41.462828 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:42.234539 (+771711us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b2c00032a08. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:42.235939 (+ 1400us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.100197 (+864258us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:43.100237 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.100313 (+ 76us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.532588 (+432275us) raft_consensus.cc:1671] finished
0812 19:07:43.532609 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.532806 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":744063,"raft.queue_time_us":744609,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":183,"spinlock_wait_cycles":25590144,"thread_start_us":839743,"threads_started":1,"wal-append.queue_time_us":1041349}
I20260812 19:07:43.551082 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.551903 29579 raft_consensus.cc:2396] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260812 19:07:43.549588 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.552759 29569 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:07:43.554591 29283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.520s user 0.020s sys 0.032s
W20260812 19:07:43.554220 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 58362us
I20260812 19:07:43.555989 29530 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:43.556202 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:43.539079 1187 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260812 19:07:43.560055 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.560629 29569 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:43.562191 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.563534 29579 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:43.564528 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:43.565308 29283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.531s user 0.020s sys 0.032s
W20260812 19:07:43.566962 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 449596us
I20260812 19:07:43.568511 30686 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:43.570194 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.570947 29456 raft_consensus.cc:2379] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:43.571209 29458 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260812 19:07:43.571652 29458 pending_rounds.cc:76] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 3
I20260812 19:07:43.572126 29458 pending_rounds.cc:97] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260812 19:07:43.530748 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.572222 29445 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:43.575314 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.575927 29459 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:43.572386 29458 raft_consensus.cc:2866] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:43.535682 30679 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:43.573738 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:43.576491 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.577003 29456 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:43.573254 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:43.449031 959 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:43.579849 29572 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.024s user 0.021s sys 0.027s
I20260812 19:07:43.577126 29451 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
W20260812 19:07:43.586712 29572 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.032s user 0.021s sys 0.027s
I20260812 19:07:43.584615 29569 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:07:43.554929 1136 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:43.580497 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.588344 30623 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:43.589411 29458 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:43.555428 726 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:43.590951 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.571274 777 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:43.590121 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:43.478783 1033 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:43.574321 1134 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:43.600433 29456 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:43.572997 1166 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.634730 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:43.635314 29456 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:43.598606 29450 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:43.598711 29569 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:43.573137 924 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.667186 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.669806 29451 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:43.640702 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2093) took 1127 ms. Trace:
I20260812 19:07:43.667088 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.671389 29457 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:43.591701 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.628712 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:43.673226 29445 raft_consensus.cc:2379] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:43.668967 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 554) took 2166 ms. Trace:
I20260812 19:07:43.670562 29569 rpcz_store.cc:276] 0812 19:07:42.512899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.597479 (+1084580us) service_pool.cc:227] Handling call
0812 19:07:43.598514 (+ 1035us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.640446 (+ 41932us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.640636 (+ 190us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:43.675068 29569 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:43.580120 1070 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.672480 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:43.680662 29341 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:43.684561 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2094) took 1130 ms. Trace:
I20260812 19:07:43.684964 29569 rpcz_store.cc:276] 0812 19:07:42.554495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.674596 (+1120101us) service_pool.cc:227] Handling call
0812 19:07:43.674891 (+ 295us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.684305 (+ 9414us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.684500 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:43.686254 29569 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260812 19:07:43.686800 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2095) took 1101 ms. Trace:
I20260812 19:07:43.687094 29569 rpcz_store.cc:276] 0812 19:07:42.585216 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.685672 (+1100456us) service_pool.cc:227] Handling call
0812 19:07:43.686027 (+ 355us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:43.686603 (+ 576us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.686757 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:43.586534 1162 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:43.719972 29332 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:43.626372 1061 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.752758 29569 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:43.765424 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2097) took 1053 ms. Trace:
I20260812 19:07:43.765765 29569 rpcz_store.cc:276] 0812 19:07:42.711928 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.748316 (+1036388us) service_pool.cc:227] Handling call
0812 19:07:43.752406 (+ 4090us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.765137 (+ 12731us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.765365 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:43.756363 29331 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:43.593081 726 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:43.760601 29452 raft_consensus.cc:3032] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:43.787292 929 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:43.787633 29452 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:43.625638 1074 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.625255 777 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.774603 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:43.675910 29578 rpcz_store.cc:276] 0812 19:07:41.503079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.504881 (+ 1802us) service_pool.cc:227] Handling call
0812 19:07:41.534308 (+ 29427us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:41.534520 (+ 212us) raft_consensus.cc:1520] Early marking committed up to index 5
0812 19:07:41.984505 (+449985us) log.cc:842] Serialized 14 byte log entry
0812 19:07:43.668645 (+1684140us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:43.668692 (+ 47us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:43.668745 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.668806 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.668909 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1104241,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":174883,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3329,"thread_start_us":2096755,"threads_started":3,"wal-append.queue_time_us":1416493}
I20260812 19:07:43.557019 1083 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:43.773254 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:43.838899 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 76545.1us
I20260812 19:07:43.669149 1134 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:43.797580 29326 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:43.599195 1036 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:43.759953 29456 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
W20260812 19:07:43.922566 29580 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.028s sys 0.059s
I20260812 19:07:43.673424 1155 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.682446 1012 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.667125 708 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:43.831146 29452 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:43.668025 533 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:43.675462 1166 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:43.682639 924 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:44.098865 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 52629.6us
W20260812 19:07:44.350528 29580 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s user 0.028s sys 0.087s
I20260812 19:07:44.351320 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 567) took 2226 ms. Trace:
W20260812 19:07:44.308959 29567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2086) took 2303 ms (client timeout 1902 ms). Trace:
W20260812 19:07:44.352892 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.644s user 0.007s sys 0.015s
I20260812 19:07:43.713449 928 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:43.970252 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.788s user 0.021s sys 0.042s
W20260812 19:07:44.399400 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.222s user 0.031s sys 0.051s
W20260812 19:07:44.158401 29282 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.007s sys 0.008s
W20260812 19:07:44.392803 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.325s user 0.008s sys 0.016s
I20260812 19:07:44.351809 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:44.037014 29403 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.833s user 0.008s sys 0.060s
W20260812 19:07:44.400241 29282 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.760s user 0.019s sys 0.008s
W20260812 19:07:44.400486 29403 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.196s user 0.014s sys 0.071s
I20260812 19:07:44.351984 29580 rpcz_store.cc:276] 0812 19:07:42.125305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.127676 (+ 2371us) service_pool.cc:227] Handling call
0812 19:07:42.128060 (+ 384us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.128242 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:43.393555 (+1265313us) log.cc:842] Serialized 14 byte log entry
0812 19:07:44.351015 (+957460us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.351064 (+ 49us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:44.351112 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.351170 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.351270 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":510819,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":2220285,"threads_started":2}
W20260812 19:07:44.400578 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 760287us
I20260812 19:07:44.401604 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:44.272505 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 568) took 2120 ms. Trace:
I20260812 19:07:44.082963 29578 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
W20260812 19:07:44.164461 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 449657us
I20260812 19:07:44.308115 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2332) took 4431 ms. Trace:
I20260812 19:07:44.251214 29355 tablet_copy_service.cc:342] P 6dd313c6592d4f33bab314a73f28b007: Request end of tablet copy session a34c47dd29c14748a018d716ac6c39df-005defc807dc47d881add64226efb7ce received from {username='slave'} at 127.0.0.1:41568
W20260812 19:07:44.396147 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 445010us
W20260812 19:07:44.396373 1188 log.cc:924] Time spent T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.677s user 0.000s sys 0.000s
I20260812 19:07:44.402710 29453 rpcz_store.cc:276] 0812 19:07:39.876984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.884393 (+ 7409us) service_pool.cc:227] Handling call
0812 19:07:42.464691 (+2580298us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:42.465488 (+ 797us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.307853 (+1842365us) spinlock_profiling.cc:243] Waited 2.53 s on lock 0x7b54003f4150. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.308009 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5826560896}
I20260812 19:07:44.408492 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.409250 1154 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:44.119087 702 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s user 0.006s sys 0.038s
I20260812 19:07:44.403712 29456 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:44.405591 29342 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.404578 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 576) took 1628 ms. Trace:
I20260812 19:07:43.832049 726 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:44.350759 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:44.253263 30685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 789) took 3552 ms (client timeout 2397 ms). Trace:
W20260812 19:07:44.408710 980 consensus_meta.cc:287] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 2.229s user 0.015s sys 0.007s
I20260812 19:07:44.409861 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.410312 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2244) took 1886 ms. Trace:
I20260812 19:07:44.352974 29452 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:44.089602 1177 log.cc:924] Time spent T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.301s user 0.000s sys 0.000s
W20260812 19:07:44.399947 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 1.22236e+06us
W20260812 19:07:44.402752 29567 rpcz_store.cc:269] 0812 19:07:42.033354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.038511 (+ 5157us) service_pool.cc:227] Handling call
0812 19:07:44.308768 (+2270257us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000aee54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.308880 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2677692288}
I20260812 19:07:44.299547 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 781) took 1162 ms. Trace:
I20260812 19:07:44.416373 1083 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.439175 1083 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:44.439519 29527 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:44.439644 1083 raft_consensus.cc:686] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:44.440603 1083 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.433251 29337 rpcz_store.cc:276] 0812 19:07:42.524424 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.589731 (+ 65307us) service_pool.cc:227] Handling call
0812 19:07:42.590127 (+ 396us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:42.595926 (+ 5799us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.595959 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:42.596427 (+ 468us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.361569 (+765142us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:43.361606 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.361671 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.410076 (+1048405us) raft_consensus.cc:1671] finished
0812 19:07:44.410105 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.410253 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8335,"raft.queue_time_us":2993,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":68551,"thread_start_us":674633,"threads_started":1,"wal-append.queue_time_us":902559}
I20260812 19:07:44.412365 29458 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:44.403488 1034 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:44.417104 1204 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.146s user 0.000s sys 0.000s
W20260812 19:07:44.418262 1183 log.cc:924] Time spent T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.384s user 0.000s sys 0.001s
I20260812 19:07:44.442394 1034 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:44.432025 702 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s user 0.009s sys 0.048s
I20260812 19:07:44.443856 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 564) took 2386 ms. Trace:
I20260812 19:07:44.444792 1034 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.444204 29576 rpcz_store.cc:276] 0812 19:07:42.057731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.058290 (+ 559us) service_pool.cc:227] Handling call
0812 19:07:42.112935 (+ 54645us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:42.144574 (+ 31639us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.144606 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:42.499158 (+354552us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.529859 (+1030701us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:43.530180 (+ 321us) log.cc:842] Serialized 14 byte log entry
0812 19:07:43.530358 (+ 178us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.530427 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.443670 (+913243us) raft_consensus.cc:1671] finished
0812 19:07:44.443690 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.443804 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33248,"prepare.queue_time_us":3758,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":36947,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":268,"thread_start_us":959783,"threads_started":1,"wal-append.queue_time_us":1098395}
I20260812 19:07:44.421605 912 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.352865 1134 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:44.407555 29355 tablet_copy_service.cc:433] P 6dd313c6592d4f33bab314a73f28b007: ending tablet copy session a34c47dd29c14748a018d716ac6c39df-005defc807dc47d881add64226efb7ce on tablet 005defc807dc47d881add64226efb7ce with peer a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:44.453006 1017 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.343s user 0.010s sys 0.000s
I20260812 19:07:44.453405 1017 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.455442 1017 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.456104 30681 raft_consensus.cc:2354] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:44.457530 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 778) took 1345 ms. Trace:
I20260812 19:07:44.430824 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 575) took 1667 ms. Trace:
I20260812 19:07:44.458209 29574 rpcz_store.cc:276] 0812 19:07:42.763217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.837916 (+1074699us) service_pool.cc:227] Handling call
0812 19:07:44.195745 (+357829us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.347935 (+152190us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.353203 (+ 5268us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.399704 (+ 46501us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:44.404006 (+ 4302us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.412751 (+ 8745us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.430754 (+ 18003us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":62152,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":51529}
I20260812 19:07:44.459735 1160 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.459786 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:43.518230 30676 rpcz_store.cc:276] 0812 19:07:41.329315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.332165 (+ 2850us) service_pool.cc:227] Handling call
0812 19:07:41.953795 (+621630us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:41.954088 (+ 293us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:42.032108 (+ 78020us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.033944 (+ 1836us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:42.129851 (+ 95907us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.212708 (+1082857us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:43.212748 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.212821 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:43.409160 (+196339us) raft_consensus.cc:1671] finished
0812 19:07:43.409178 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:43.409428 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":27675008,"thread_start_us":978454,"threads_started":1,"wal-append.queue_time_us":1259887}
I20260812 19:07:44.461627 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.462517 30676 raft_consensus.cc:2396] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:07:44.431279 1154 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.463488 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.422298 1102 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:44.423785 1093 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:44.414791 29578 rpcz_store.cc:276] 0812 19:07:42.776568 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.840092 (+1063524us) service_pool.cc:227] Handling call
0812 19:07:44.082764 (+242672us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.404340 (+321576us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.404522 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:44.470086 29334 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.844s user 0.009s sys 0.000s
I20260812 19:07:44.425927 1166 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.428543 1000 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:44.430212 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:44.430521 936 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.934s user 0.008s sys 0.006s
I20260812 19:07:44.430306 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:44.260942 30674 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.199491 1202 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:43.923736 777 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:44.433068 29453 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:44.399426 1012 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:44.432909 30685 rpcz_store.cc:269] 0812 19:07:40.700735 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.762958 (+ 62223us) service_pool.cc:227] Handling call
0812 19:07:42.451428 (+1688470us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:44.253045 (+1801617us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54004605d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.253179 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1376411776}
I20260812 19:07:44.436206 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 577) took 1644 ms. Trace:
I20260812 19:07:44.437713 29342 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.438012 30671 rpcz_store.cc:276] 0812 19:07:43.137634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.499327 (+361693us) service_pool.cc:227] Handling call
0812 19:07:43.499647 (+ 320us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.541400 (+ 41753us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:43.541432 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:43.541472 (+ 40us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:43.541510 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.299303 (+757793us) spinlock_profiling.cc:243] Waited 41.5 ms on lock 0x7b54007526d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.299348 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.299481 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":95516416}
I20260812 19:07:44.517679 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.446700 29527 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:44.519965 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 79416.8us
I20260812 19:07:43.790436 825 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.503317 702 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.521869 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 786) took 3822 ms. Trace:
I20260812 19:07:44.511335 1202 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.522327 30674 rpcz_store.cc:276] 0812 19:07:40.699846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:40.832277 (+132431us) service_pool.cc:227] Handling call
0812 19:07:43.154338 (+2322061us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66ebabce 000000000034aea9 00007f9c6ecdaa30 00007f9c6eda07b4 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:43.163948 (+ 9610us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.260737 (+1096789us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b540046b7a0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d9e730 00007f9c69e10300 00007f9c69e11371 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc
0812 19:07:44.502626 (+241889us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.521799 (+ 19173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":146166912}
I20260812 19:07:44.524859 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:44.525343 1136 consensus_meta.cc:287] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.900s user 0.005s sys 0.001s
I20260812 19:07:44.476053 1102 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.526156 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.526804 30674 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:44.527953 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.528542 30674 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:44.529433 29569 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.476544 1093 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.522652 702 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.530766 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2102) took 1769 ms. Trace:
I20260812 19:07:44.531103 29569 rpcz_store.cc:276] 0812 19:07:42.761829 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.787278 (+1025449us) service_pool.cc:227] Handling call
0812 19:07:44.310209 (+522931us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.529805 (+219596us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.530566 (+ 761us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400262854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.530715 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":503745920}
I20260812 19:07:44.531765 1093 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.532114 702 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.533149 30686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 743) took 1278 ms. Trace:
I20260812 19:07:44.448225 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.533488 30686 rpcz_store.cc:276] 0812 19:07:43.255079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.569500 (+314421us) service_pool.cc:227] Handling call
0812 19:07:43.588258 (+ 18758us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:43.592892 (+ 4634us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:43.592922 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:43.624026 (+ 31104us) log.cc:842] Serialized 26 byte log entry
0812 19:07:44.057012 (+432986us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:44.057053 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.057133 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.532975 (+475842us) raft_consensus.cc:1671] finished
0812 19:07:44.532993 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.533099 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":683322,"raft.queue_time_us":4332,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":69923,"thread_start_us":432593,"threads_started":1,"wal-append.queue_time_us":903238}
I20260812 19:07:44.534276 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:44.413506 1023 consensus_meta.cc:287] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.373s user 0.009s sys 0.015s
I20260812 19:07:44.536177 1023 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:44.449383 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44348.7us
I20260812 19:07:44.536804 29527 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:44.526458 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.537636 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 571) took 2378 ms. Trace:
I20260812 19:07:44.537817 29453 raft_consensus.cc:2396] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:07:44.514963 29580 rpcz_store.cc:276] 0812 19:07:42.791732 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.402384 (+1610652us) service_pool.cc:227] Handling call
0812 19:07:44.435880 (+ 33496us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.436008 (+ 128us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.436022 (+ 14us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.436047 (+ 25us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:44.436070 (+ 23us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.436103 (+ 33us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.436165 (+ 62us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.538483 29579 rpcz_store.cc:276] 0812 19:07:42.159605 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.572610 (+1413005us) service_pool.cc:227] Handling call
0812 19:07:43.572977 (+ 367us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:43.578247 (+ 5270us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:43.578279 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:43.578702 (+ 423us) log.cc:842] Serialized 26 byte log entry
0812 19:07:44.057390 (+478688us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:44.057432 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.057501 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.537433 (+479932us) raft_consensus.cc:1671] finished
0812 19:07:44.537453 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.537582 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9485,"thread_start_us":389674,"threads_started":1,"wal-append.queue_time_us":943157}
I20260812 19:07:44.399466 1155 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.448654 533 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:44.450791 1063 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:44.465209 29452 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:44.424995 512 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.457846 30681 rpcz_store.cc:276] 0812 19:07:43.111831 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.112441 (+ 610us) service_pool.cc:227] Handling call
0812 19:07:44.457326 (+1344885us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54005104d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.457459 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3087682304}
I20260812 19:07:44.560195 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.471364 29334 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:43.783897 1070 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.474316 29282 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:44.460789 1160 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:44.568977 1160 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:44.569428 1160 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:44.563210 512 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.569968 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.571171 512 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.571673 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.573352 1102 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.573979 1081 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:44.530109 1154 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.486912 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:44.080791 1074 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.577028 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.575624 29576 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:43.922123 929 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.578393 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2061) took 1661 ms. Trace:
I20260812 19:07:44.578714 29576 rpcz_store.cc:276] 0812 19:07:42.917470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.575161 (+1657691us) service_pool.cc:227] Handling call
0812 19:07:44.575453 (+ 292us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.578150 (+ 2697us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.578338 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.579218 1023 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.579803 29575 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.581009 745 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.581279 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 569) took 2429 ms. Trace:
I20260812 19:07:44.484001 29456 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:44.432785 29446 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:44.582310 533 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.157596 1207 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.341s
W20260812 19:07:44.591696 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 839551us
I20260812 19:07:44.524802 1202 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.466216 959 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.443956 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.526849 1209 raft_consensus.cc:2741] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260812 19:07:44.598111 29567 raft_consensus.cc:2396] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:07:44.406960 29572 rpcz_store.cc:276] 0812 19:07:42.152019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.157541 (+ 5522us) service_pool.cc:227] Handling call
0812 19:07:42.157883 (+ 342us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:42.158279 (+ 396us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:42.158306 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:42.498002 (+339696us) log.cc:842] Serialized 52 byte log entry
0812 19:07:43.587215 (+1089213us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:43.587254 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:43.587317 (+ 63us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.272251 (+684934us) raft_consensus.cc:1671] finished
0812 19:07:44.272275 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.272435 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2751,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":5288,"thread_start_us":1031999,"threads_started":1,"wal-append.queue_time_us":1638408}
I20260812 19:07:44.540082 29281 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:44.532138 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:44.600011 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 60163.1us
I20260812 19:07:44.525741 1136 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.550621 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.173528 1152 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.577622 1074 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.548450 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2103) took 1710 ms. Trace:
I20260812 19:07:44.632148 29579 rpcz_store.cc:276] 0812 19:07:42.837885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.540197 (+1702312us) service_pool.cc:227] Handling call
0812 19:07:44.540534 (+ 337us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.540750 (+ 216us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.540777 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.548173 (+ 7396us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:44.548228 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.548289 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.548396 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.633369 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.579892 929 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.411839 928 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.555105 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.637517 29337 raft_consensus.cc:2396] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260812 19:07:44.553391 29452 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260812 19:07:44.582662 30679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 734) took 1665 ms (client timeout 2096 ms). Trace:
W20260812 19:07:44.565191 948 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260812 19:07:44.565176 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:44.565531 1025 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:07:44.567381 1070 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.545995 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 744) took 1261 ms. Trace:
I20260812 19:07:44.502836 936 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.577677 29580 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:44.565493 1208 consensus_queue.cc:1046] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260812 19:07:44.503940 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.584481 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:44.653908 29342 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:43.840297 1061 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.661854 1061 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.647514 30670 raft_consensus.cc:1226] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:44.664175 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2144) took 1933 ms. Trace:
I20260812 19:07:44.664547 29338 rpcz_store.cc:276] 0812 19:07:42.730465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.730585 (+ 120us) service_pool.cc:227] Handling call
0812 19:07:42.740263 (+ 9678us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.531225 (+790962us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.663946 (+1132721us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54002323d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.664116 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1631340288}
I20260812 19:07:44.581670 29575 rpcz_store.cc:276] 0812 19:07:42.152349 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.208390 (+ 56041us) service_pool.cc:227] Handling call
0812 19:07:42.261626 (+ 53236us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.580241 (+2318615us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.581048 (+ 807us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b540012be54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.581216 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5294644992}
I20260812 19:07:44.631170 1205 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:44.649516 1154 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.666257 1205 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.666419 1154 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.668365 1205 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.668730 1154 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.670523 1208 raft_consensus.cc:3032] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:44.670660 1152 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.671280 1152 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.672343 936 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.602596 29574 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:44.640501 1202 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.643141 702 consensus_queue.cc:1046] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260812 19:07:44.582653 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.640727 1209 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:44.673641 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2361) took 1094 ms. Trace:
I20260812 19:07:44.672576 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2363) took 1091 ms. Trace:
I20260812 19:07:44.673164 1152 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.675565 1155 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.676497 29456 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:07:44.677486 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.678077 29456 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:44.678856 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.665262 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 740) took 1538 ms. Trace:
I20260812 19:07:44.679800 30670 rpcz_store.cc:276] 0812 19:07:43.126625 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.424084 (+297459us) service_pool.cc:227] Handling call
0812 19:07:43.458367 (+ 34283us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.664184 (+1205817us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.665006 (+ 822us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005eecd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.665203 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2712850688}
I20260812 19:07:44.354449 708 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.589927 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.675971 29458 rpcz_store.cc:276] 0812 19:07:43.581656 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.590621 (+ 8965us) service_pool.cc:227] Handling call
0812 19:07:44.672412 (+1081791us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b5400212354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.672516 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1881637120}
I20260812 19:07:44.679884 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.544499 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.567951 1196 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:44.599489 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.603641 686 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:44.604288 932 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:44.598059 980 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.604984 1012 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.608044 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2357) took 1030 ms. Trace:
I20260812 19:07:44.610181 29572 raft_consensus.cc:1226] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:44.521493 30671 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:07:44.691701 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 798) took 1014 ms. Trace:
I20260812 19:07:44.636641 29579 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:44.692823 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 578) took 1512 ms. Trace:
I20260812 19:07:44.693149 29579 rpcz_store.cc:276] 0812 19:07:43.180748 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.635029 (+1454281us) service_pool.cc:227] Handling call
0812 19:07:44.635313 (+ 284us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.692581 (+ 57268us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.692776 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.638237 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1911) took 1046 ms. Trace:
W20260812 19:07:44.563817 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 89566.2us
I20260812 19:07:44.692044 30671 rpcz_store.cc:276] 0812 19:07:43.676807 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.517392 (+840585us) service_pool.cc:227] Handling call
0812 19:07:44.691640 (+174248us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.695129 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.598326 924 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.648350 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:44.693778 29452 rpcz_store.cc:276] 0812 19:07:43.591967 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.552813 (+960846us) service_pool.cc:227] Handling call
0812 19:07:44.553086 (+ 273us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.637962 (+ 84876us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.638179 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.553934 1134 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.582911 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2362) took 1003 ms. Trace:
I20260812 19:07:44.650285 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:44.702677 29579 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:44.650964 29527 leader_election.cc:398] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:44.652112 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.649346 30674 rpcz_store.cc:276] 0812 19:07:43.268546 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.529163 (+1260617us) service_pool.cc:227] Handling call
0812 19:07:44.529467 (+ 304us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.529656 (+ 189us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.529683 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.529725 (+ 42us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:44.529760 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.529824 (+ 64us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.529922 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.657620 30676 raft_consensus.cc:2396] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260812 19:07:44.641374 30679 rpcz_store.cc:269] 0812 19:07:42.917063 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.917994 (+ 931us) service_pool.cc:227] Handling call
0812 19:07:44.582457 (+1664463us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b540046bc54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:44.582565 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":909651200}
I20260812 19:07:44.660174 1060 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:44.674422 29575 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260812 19:07:44.682126 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:44.538262 825 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:44.675325 29457 rpcz_store.cc:276] 0812 19:07:43.578797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.672027 (+ 93230us) service_pool.cc:227] Handling call
0812 19:07:44.672962 (+1000935us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:44.673172 (+ 210us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.673349 (+ 177us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.673374 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.673412 (+ 38us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:44.673448 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.673500 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.673590 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":82769152}
I20260812 19:07:44.686252 30684 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:44.715868 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.695997 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.696717 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:44.778491 29331 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 4
I20260812 19:07:44.681205 1208 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:44.674551 702 raft_consensus.cc:3032] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:44.700037 29578 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:44.819074 29569 raft_consensus.cc:2379] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:07:44.704505 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 53683.8us
I20260812 19:07:44.676206 912 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:44.690096 932 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:44.705516 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.674474 29574 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:44.673511 1202 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.675282 1209 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.706161 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:44.707038 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:44.708053 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2105) took 1604 ms. Trace:
I20260812 19:07:44.854878 29572 rpcz_store.cc:276] 0812 19:07:43.104001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.608175 (+1504174us) service_pool.cc:227] Handling call
0812 19:07:44.608477 (+ 302us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.707818 (+ 99341us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.707996 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.708192 928 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:44.708953 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.690578 29459 rpcz_store.cc:276] 0812 19:07:43.577669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.589109 (+ 11440us) service_pool.cc:227] Handling call
0812 19:07:43.667785 (+ 78676us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:43.668189 (+ 404us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:43.668218 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:43.684146 (+ 15928us) log.cc:842] Serialized 26 byte log entry
0812 19:07:43.972011 (+287865us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:44.503447 (+531436us) log.cc:842] Serialized 14 byte log entry
0812 19:07:44.504167 (+ 720us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.504376 (+ 209us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:44.607857 (+103481us) raft_consensus.cc:1671] finished
0812 19:07:44.607874 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.607990 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":21241,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":76826,"thread_start_us":584801,"threads_started":2,"wal-append.queue_time_us":817365}
I20260812 19:07:44.710532 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.710609 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.698910 726 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:44.864078 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2366) took 1281 ms. Trace:
I20260812 19:07:44.864602 29450 rpcz_store.cc:276] 0812 19:07:43.582457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.667076 (+ 84619us) service_pool.cc:227] Handling call
0812 19:07:44.710389 (+1043313us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:44.863981 (+153592us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5258112}
I20260812 19:07:44.711963 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2364) took 1130 ms. Trace:
I20260812 19:07:44.865648 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.866026 29445 rpcz_store.cc:276] 0812 19:07:43.581971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.673785 (+ 91814us) service_pool.cc:227] Handling call
0812 19:07:44.711285 (+1037500us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:44.711482 (+ 197us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.711672 (+ 190us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.711695 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.711732 (+ 37us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:44.711767 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.711815 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:44.711907 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":82883456}
I20260812 19:07:44.867419 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.868227 29445 raft_consensus.cc:2396] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:07:44.712442 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:44.713310 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 63111.1us
I20260812 19:07:44.869680 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.701215 29446 rpcz_store.cc:276] 0812 19:07:43.579074 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.626407 (+ 47333us) service_pool.cc:227] Handling call
0812 19:07:44.582850 (+956443us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.712821 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 638) took 5307 ms. Trace:
I20260812 19:07:44.871757 29326 rpcz_store.cc:276] 0812 19:07:39.405474 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:39.405602 (+ 128us) service_pool.cc:227] Handling call
0812 19:07:39.412070 (+ 6468us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.350296 (+4938226us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:44.712587 (+362291us) spinlock_profiling.cc:243] Waited 4.34 s on lock 0x7b540033a254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:44.712738 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9972615424}
I20260812 19:07:44.649427 929 consensus_queue.cc:1046] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260812 19:07:44.781764 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.892207 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:44.892642 29458 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:44.682034 1162 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:44.786469 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.810485 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.810833 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:44.814342 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:44.815692 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.816471 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.817126 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.817970 702 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:44.826169 29176 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.699s user 0.019s sys 0.057s
I20260812 19:07:44.727257 1078 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:07:44.874070 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.880440 928 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:44.713826 980 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:44.865648 726 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:45.046686 29176 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.927s user 0.024s sys 0.073s
I20260812 19:07:45.048283 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:44.930104 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:44.894414 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:44.864111 29452 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:44.819849 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 796) took 1148 ms. Trace:
I20260812 19:07:45.049391 30681 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:45.051816 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:44.854633 811 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s user 0.017s sys 0.012s
I20260812 19:07:44.957064 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.051048 29453 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:44.826592 932 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.015170 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:45.063088 29459 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:45.062958 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:44.797202 29331 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
W20260812 19:07:45.029194 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.627s user 0.059s sys 0.080s
I20260812 19:07:45.063681 29445 raft_consensus.cc:2362] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3. Re-sending same reply.
W20260812 19:07:45.064081 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.662s user 0.060s sys 0.081s
I20260812 19:07:45.047940 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1912) took 1447 ms. Trace:
I20260812 19:07:45.054013 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:45.064638 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 663081us
I20260812 19:07:45.067230 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.046542 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:45.068796 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.069274 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:45.066421 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1913) took 1438 ms. Trace:
I20260812 19:07:45.069942 29450 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:45.068538 29456 rpcz_store.cc:276] 0812 19:07:43.600868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.047072 (+1446204us) service_pool.cc:227] Handling call
0812 19:07:45.047381 (+ 309us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.047559 (+ 178us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.047583 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.047623 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:45.047659 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.047709 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.047886 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.066363 29282 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.067821 30681 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:45.080750 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.064386 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:45.070722 29457 rpcz_store.cc:276] 0812 19:07:43.628307 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.059996 (+1431689us) service_pool.cc:227] Handling call
0812 19:07:45.065924 (+ 5928us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.066130 (+ 206us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.066153 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.066192 (+ 39us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:45.066224 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.066274 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.066368 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:44.956221 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:45.083678 29446 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:45.085407 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.092142 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.064409 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.065433 1074 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:45.096040 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.098052 29528 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:45.101325 29399 connection.cc:232] RPC call timeout handler was delayed by -1.72133152699098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2736s for a response
W20260812 19:07:45.062024 811 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.970s user 0.020s sys 0.022s
I20260812 19:07:45.052516 1078 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:45.001765 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.107154 29527 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:45.070212 1034 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s user 0.021s sys 0.018s
I20260812 19:07:45.001572 30670 raft_consensus.cc:2354] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:44.959345 1206 raft_consensus.cc:1050] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007: attempting to promote NON_VOTER a34c47dd29c14748a018d716ac6c39df to VOTER
I20260812 19:07:44.958051 1202 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.105883 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:45.103940 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.116089 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1916) took 1278 ms. Trace:
I20260812 19:07:45.111524 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1915) took 1322 ms. Trace:
W20260812 19:07:45.111879 29334 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.039s sys 0.062s
I20260812 19:07:45.103412 1074 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:45.114434 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.116329 29450 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:45.117034 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:45.126955 29334 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.529s user 0.039s sys 0.062s
I20260812 19:07:44.963461 1136 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:44.849584 912 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.119082 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.119194 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:44.862351 706 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.246s
I20260812 19:07:44.931077 1060 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.177s
I20260812 19:07:45.098666 1014 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260812 19:07:45.116885 29403 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.702s user 0.027s sys 0.027s
I20260812 19:07:44.961336 1209 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:45.165652 29403 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.750s user 0.027s sys 0.027s
I20260812 19:07:45.109040 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:44.964376 1106 consensus_queue.cc:1046] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260812 19:07:45.062479 30676 rpcz_store.cc:276] 0812 19:07:43.671793 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.489998 (+818205us) service_pool.cc:227] Handling call
0812 19:07:44.784071 (+294073us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:45.113075 1034 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.582s user 0.021s sys 0.018s
I20260812 19:07:45.126477 29453 rpcz_store.cc:276] 0812 19:07:43.789575 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.078808 (+1289233us) service_pool.cc:227] Handling call
0812 19:07:45.107747 (+ 28939us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.111188 (+ 3441us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.111217 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.111257 (+ 40us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:45.111297 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.111354 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.111470 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3034,"raft.queue_time_us":6785}
I20260812 19:07:45.126312 29459 rpcz_store.cc:276] 0812 19:07:43.925079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.080867 (+1155788us) service_pool.cc:227] Handling call
0812 19:07:45.112516 (+ 31649us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.115775 (+ 3259us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.115801 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.115837 (+ 36us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:45.115877 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.115936 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.116034 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2914,"raft.queue_time_us":5377}
I20260812 19:07:45.046535 29572 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:45.107492 1076 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:45.110890 1078 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:45.121879 30679 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:45.114060 1152 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:45.145037 29457 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260812 19:07:45.162973 1134 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.063342 1162 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.027642 777 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.152994 30625 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:45.163954 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 44960.2us
W20260812 19:07:45.166041 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 750843us
I20260812 19:07:45.196118 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:45.197419 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:45.197731 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.111249 1155 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260812 19:07:45.130455 1074 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:45.175951 29446 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:45.047860 1208 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.176782 30681 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:45.184444 29456 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.180660 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.203729 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:45.205324 29457 raft_consensus.cc:595] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.162997 1129 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:45.118847 29574 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:45.167958 708 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:45.214452 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 108636us
I20260812 19:07:45.167968 825 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.166438 30671 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:45.220245 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.220894 30671 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:45.165304 1014 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:45.185760 1106 raft_consensus.cc:3032] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:45.173074 29450 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.180373 1209 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:45.203435 29572 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:45.206274 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.205116 29567 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:45.180713 1196 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:45.187597 763 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:45.205066 745 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:45.210855 29341 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.211297 1208 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:45.204299 1152 raft_consensus.cc:686] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:45.212801 777 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.213328 29577 raft_consensus.cc:1226] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
W20260812 19:07:45.213492 29580 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s user 0.000s sys 0.015s
W20260812 19:07:45.257733 29580 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.600s user 0.000s sys 0.015s
I20260812 19:07:45.183241 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.235149 1136 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:45.131132 702 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:45.213967 1102 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s user 0.001s sys 0.013s
I20260812 19:07:45.242398 30625 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.243332 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.273162 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.245093 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260812 19:07:45.275399 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:45.276085 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:45.247887 29339 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:07:45.216575 726 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:45.207816 1134 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:45.204428 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.248311 1196 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:45.250684 709 consensus_queue.cc:1046] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 19:07:45.236356 1160 consensus_meta.cc:287] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.666s user 0.010s sys 0.000s
I20260812 19:07:45.208036 1129 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:45.257874 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 748) took 1714 ms. Trace:
I20260812 19:07:45.291487 1129 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:45.293303 998 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.237618 1014 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:45.223894 29342 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.570s user 0.008s sys 0.000s
I20260812 19:07:45.294672 1081 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:45.295030 29342 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:45.266916 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:07:45.215294 1083 consensus_queue.cc:1159] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "182784ca1eb74135b8f16d8894fa4b01" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260812 19:07:45.296561 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.249859 745 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:45.238152 1106 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:45.241474 30675 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:45.271337 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2131) took 4557 ms. Trace:
I20260812 19:07:45.249017 1205 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:45.249374 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:45.289563 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 47259.6us
I20260812 19:07:45.241142 1209 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:45.298663 30629 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:45.288071 1239 log.cc:924] Time spent T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260812 19:07:45.298856 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.297499 30671 rpcz_store.cc:276] 0812 19:07:43.543354 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.238555 (+1695201us) service_pool.cc:227] Handling call
0812 19:07:45.257359 (+ 18804us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.257568 (+ 209us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.257591 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.257630 (+ 39us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:45.257660 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.257707 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.257817 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:45.287593 1102 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.614s user 0.002s sys 0.013s
I20260812 19:07:45.305181 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.302860 29335 rpcz_store.cc:276] 0812 19:07:40.713822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.146997 (+433175us) service_pool.cc:227] Handling call
0812 19:07:41.147410 (+ 413us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:43.178560 (+2031150us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.270126 (+2091566us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b5400180bd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:45.271261 (+ 1135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4618565504}
I20260812 19:07:45.268616 980 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.306783 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.314468 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.315299 29567 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:45.313515 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:45.316645 1209 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:45.317580 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.297608 811 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:07:45.320371 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 254077us
I20260812 19:07:45.299978 745 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.320814 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.295073 1081 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:45.326781 29567 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:45.271523 1136 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.332823 30671 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:45.333501 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 790) took 1664 ms. Trace:
I20260812 19:07:45.333818 30671 rpcz_store.cc:276] 0812 19:07:43.669138 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.309232 (+1640094us) service_pool.cc:227] Handling call
0812 19:07:45.327327 (+ 18095us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.333249 (+ 5922us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.333444 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.335845 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.336295 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.336678 29572 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:45.328164 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.303084 30676 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
W20260812 19:07:45.325340 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 245692us
I20260812 19:07:45.291435 1160 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.340714 998 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:45.320647 709 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 19:07:45.313347 928 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:07:45.353039 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 245937us
I20260812 19:07:45.307184 726 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260812 19:07:45.366556 30671 raft_consensus.cc:1226] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:45.367220 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 793) took 1697 ms. Trace:
I20260812 19:07:45.368525 30671 rpcz_store.cc:276] 0812 19:07:43.669988 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.334740 (+1664752us) service_pool.cc:227] Handling call
0812 19:07:45.366375 (+ 31635us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.367003 (+ 628us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.367170 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.373154 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.350994 29567 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.352335 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:45.374878 29457 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:45.377570 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.330719 912 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.380965 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.326165 1205 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:45.379091 29576 raft_consensus.cc:1226] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:45.362747 1237 log.cc:924] Time spent T 6469507c6875443ebd9d4c40bb47a54a P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.198s user 0.000s sys 0.001s
I20260812 19:07:45.381337 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:45.383082 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 291740us
I20260812 19:07:45.344485 929 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:45.386232 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.352285 811 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:45.387305 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.390169 29332 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:45.392349 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.383538 30679 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:45.396585 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.399714 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.400699 30671 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:45.399416 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.366178 998 raft_consensus.cc:686] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:45.405591 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.384336 856 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.394879 1160 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.404270 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.381709 928 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:45.408025 29281 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.408226 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.340740 1106 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.409791 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.406446 29335 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
W20260812 19:07:45.398298 1249 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.090s user 0.001s sys 0.001s
I20260812 19:07:45.414047 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1914) took 1744 ms. Trace:
I20260812 19:07:45.419720 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.338387 1152 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.422971 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.425623 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.428110 29567 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:45.381564 1176 consensus_queue.cc:1046] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260812 19:07:45.419137 29452 rpcz_store.cc:276] 0812 19:07:43.669825 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.062313 (+1392488us) service_pool.cc:227] Handling call
0812 19:07:45.105067 (+ 42754us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.105278 (+ 211us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:45.132462 (+ 27184us) log.cc:842] Serialized 14 byte log entry
0812 19:07:45.413080 (+280618us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.413136 (+ 56us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:45.413189 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.413255 (+ 66us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.413985 (+ 730us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8222,"prepare.queue_time_us":82072,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":37,"raft.queue_time_us":308,"thread_start_us":279949,"threads_started":1}
I20260812 19:07:45.431820 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.407346 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 792) took 1737 ms. Trace:
I20260812 19:07:45.431428 29338 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:45.435495 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.390235 1205 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.400209 912 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.027s
I20260812 19:07:45.426687 30676 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.444267 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:45.440009 29282 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.427810 856 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.399049 811 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.458626 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:45.473017 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 371130us
W20260812 19:07:45.473583 29399 connection.cc:232] RPC call timeout handler was delayed by -1.5800598419737071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20520000000000002s for a response
I20260812 19:07:45.474407 856 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:45.474848 856 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:45.475183 856 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:45.475551 29282 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.476639 29282 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.477577 922 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:45.478468 1034 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:45.479008 1034 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:45.479422 1034 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.481266 924 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:45.481796 924 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.478013 922 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:45.482887 922 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.484315 924 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.486227 1070 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:45.486804 1070 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.487851 763 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:45.488458 763 raft_consensus.cc:686] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:45.489070 1070 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.489336 763 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.490346 856 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.493641 856 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.437472 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 584) took 1605 ms. Trace:
I20260812 19:07:45.492549 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.495226 29580 rpcz_store.cc:276] 0812 19:07:43.832514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.649981 (+817467us) service_pool.cc:227] Handling call
0812 19:07:44.656942 (+ 6961us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.657147 (+ 205us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:45.258326 (+601179us) log.cc:842] Serialized 14 byte log entry
0812 19:07:45.437143 (+178817us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.437198 (+ 55us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:45.437251 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.437313 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.437419 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":33116,"prepare.queue_time_us":467520,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2741,"raft.queue_time_us":355,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":15872,"thread_start_us":778921,"threads_started":2}
I20260812 19:07:45.500978 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.500707 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.501576 29567 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:45.426438 1193 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260812 19:07:45.505487 29530 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.509876 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.502967 29403 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:45.506853 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.387231 1196 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.443279 1257 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:45.471767 1106 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:45.512521 29530 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.507128 924 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:45.523283 29567 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:45.536994 29281 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.538703 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.508445 777 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:45.515529 1160 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:45.515882 1257 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:45.516846 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:45.555353 29453 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:07:45.545823 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 149859us
I20260812 19:07:45.513025 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 794) took 1841 ms. Trace:
I20260812 19:07:45.515722 1266 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:45.558779 30684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 782) took 2417 ms (client timeout 1976 ms). Trace:
I20260812 19:07:45.540023 1070 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260812 19:07:45.559223 30684 rpcz_store.cc:269] 0812 19:07:43.141711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.385107 (+243396us) service_pool.cc:227] Handling call
0812 19:07:45.558583 (+2173476us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54005c03d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:45.558712 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2962079104}
I20260812 19:07:45.538522 825 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:45.545897 1176 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:45.567824 30623 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.570395 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.439124 30670 rpcz_store.cc:276] 0812 19:07:43.669693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.313553 (+1643860us) service_pool.cc:227] Handling call
0812 19:07:45.344120 (+ 30567us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.366172 (+ 22052us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.378675 (+ 12503us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.383794 (+ 5119us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:45.386892 (+ 3098us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.396195 (+ 9303us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.404368 (+ 8173us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.548539 924 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:45.549758 1136 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:45.575268 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 72396.3us
I20260812 19:07:45.546248 1093 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260812 19:07:45.558698 1257 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:45.506172 998 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:45.586329 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 85699.9us
I20260812 19:07:45.566319 825 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:45.587543 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 48545.7us
I20260812 19:07:45.586711 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2360) took 2008 ms. Trace:
I20260812 19:07:45.591481 29451 rpcz_store.cc:276] 0812 19:07:43.578566 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.670370 (+ 91804us) service_pool.cc:227] Handling call
0812 19:07:44.606792 (+936422us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:44.606974 (+ 182us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:44.607317 (+ 343us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.607344 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:44.607763 (+ 419us) log.cc:842] Serialized 26 byte log entry
0812 19:07:45.093259 (+485496us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:45.112644 (+ 19385us) log.cc:842] Serialized 14 byte log entry
0812 19:07:45.116518 (+ 3874us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.116626 (+ 108us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:45.580897 (+464271us) raft_consensus.cc:1671] finished
0812 19:07:45.586516 (+ 5619us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.586654 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3415,"prepare.queue_time_us":39369,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":5478,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":88349184,"thread_start_us":440395,"threads_started":1,"wal-append.queue_time_us":660502}
I20260812 19:07:45.596267 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.576540 924 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:45.598551 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.555595 777 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.539794 928 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.578204 1176 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:45.584153 1136 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.575196 533 consensus_queue.cc:1046] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260812 19:07:45.559491 30676 rpcz_store.cc:276] 0812 19:07:43.671236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.379072 (+1707836us) service_pool.cc:227] Handling call
0812 19:07:45.405044 (+ 25972us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.465101 (+ 60057us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.509068 (+ 43967us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.618191 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2077) took 1198 ms. Trace:
I20260812 19:07:45.618618 29569 rpcz_store.cc:276] 0812 19:07:44.419439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.531922 (+112483us) service_pool.cc:227] Handling call
0812 19:07:45.618017 (+1086095us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400156654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:45.618134 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":503339904}
I20260812 19:07:45.590116 825 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:07:45.611616 1267 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260812 19:07:45.613372 1033 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.584167 1281 consensus_queue.cc:1046] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260812 19:07:45.601631 924 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:45.575706 1129 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.633744 1279 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:45.646301 30681 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:45.646871 30676 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260812 19:07:45.654834 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 78594us
I20260812 19:07:45.656622 30686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 746) took 2254 ms. Trace:
I20260812 19:07:45.656653 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.647651 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 755) took 1002 ms. Trace:
I20260812 19:07:45.657069 30686 rpcz_store.cc:276] 0812 19:07:43.402139 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.535530 (+1133391us) service_pool.cc:227] Handling call
0812 19:07:44.535836 (+ 306us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.705028 (+169192us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.705061 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:44.705103 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:44.705142 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.656405 (+951263us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54004b05d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:45.656450 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.656566 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5253,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":377630464}
I20260812 19:07:45.626756 1034 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.645391 1287 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:45.654841 1256 log.cc:924] Time spent T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.139s user 0.000s sys 0.001s
I20260812 19:07:45.649739 912 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 19:07:45.673190 29453 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:45.655174 1205 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:45.678479 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 80160.9us
W20260812 19:07:45.680279 1273 log.cc:924] Time spent T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260812 19:07:45.681073 1074 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.682008 1034 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.683225 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:45.684099 29453 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.697024 29453 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:45.669049 811 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.684484 1078 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:45.704823 1286 log.cc:924] Time spent T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Append to log took a long time: real 0.090s user 0.000s sys 0.001s
I20260812 19:07:45.705403 1078 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.703320 29342 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:45.709307 29341 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:45.688982 1136 raft_consensus.cc:3032] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:45.720139 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:45.720849 29331 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:45.663756 30676 rpcz_store.cc:276] 0812 19:07:44.644835 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.629976 (+985141us) service_pool.cc:227] Handling call
0812 19:07:45.646501 (+ 16525us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.647353 (+ 852us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.647592 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:45.725808 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:45.684113 1206 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.729635 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:45.730350 29446 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:45.664851 1176 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:45.725155 1276 log.cc:924] Time spent T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.140s user 0.000s sys 0.001s
I20260812 19:07:45.688033 1014 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.742918 29326 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.720078 1136 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:45.718998 811 consensus_queue.cc:1046] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260812 19:07:45.700141 1293 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260812 19:07:45.734648 1266 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:45.759235 1129 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.767380 1266 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:45.769057 29574 consensus_meta.cc:287] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.557s user 0.006s sys 0.006s
I20260812 19:07:45.759323 777 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.766556 1014 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:45.784761 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 149038us
W20260812 19:07:45.778921 932 consensus_meta.cc:287] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.692s user 0.012s sys 0.000s
I20260812 19:07:45.766801 1093 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:45.789808 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.776223 811 consensus_queue.cc:1225] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer dd7e8c7a8bc54e73a39411cfc241fe9c log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260812 19:07:45.785799 1166 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.281s user 0.022s sys 0.022s
I20260812 19:07:45.791904 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.780424 30676 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.785322 1266 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:45.802763 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:45.810288 30676 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260812 19:07:45.797478 1166 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.290s user 0.023s sys 0.022s
I20260812 19:07:45.791368 932 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.806782 1311 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:45.818774 1311 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.819315 29574 raft_consensus.cc:2396] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:07:45.829726 708 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:45.830330 1297 log.cc:924] Time spent T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20260812 19:07:45.831573 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:45.832185 29453 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.788341 922 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.836052 30676 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:45.794117 1093 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.830277 708 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.837456 1311 consensus_queue.cc:1046] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260812 19:07:45.783526 1136 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.842676 1136 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:45.830541 932 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.797070 1288 consensus_queue.cc:1046] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260812 19:07:45.846881 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 142888us
I20260812 19:07:45.833488 1074 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.837826 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 797) took 1272 ms. Trace:
W20260812 19:07:45.846038 1278 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.136s user 0.001s sys 0.000s
I20260812 19:07:45.857846 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 747) took 2391 ms. Trace:
I20260812 19:07:45.858721 1262 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:45.859156 1262 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:45.861207 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.862609 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.865468 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.866293 30676 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:45.868031 1262 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.873116 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.883984 29527 leader_election.cc:398] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:45.876223 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:45.884748 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:45.884985 29527 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.885440 29574 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.886227 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.858307 30674 rpcz_store.cc:276] 0812 19:07:43.466272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.956055 (+1489783us) service_pool.cc:227] Handling call
0812 19:07:44.956469 (+ 414us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:45.053851 (+ 97382us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.061359 (+ 7508us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:45.741920 (+680561us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b2c000f7778. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:45.742350 (+ 430us) log.cc:842] Serialized 26 byte log entry
0812 19:07:45.807109 (+ 64759us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:45.813393 (+ 6284us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.816189 (+ 2796us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:45.846592 (+ 30403us) raft_consensus.cc:1671] finished
0812 19:07:45.846621 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.857771 (+ 11150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":92443,"raft.queue_time_us":75262,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"spinlock_wait_cycles":37632384,"thread_start_us":53010,"threads_started":1,"wal-append.queue_time_us":98611}
W20260812 19:07:45.891235 1209 consensus_meta.cc:287] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.563s user 0.008s sys 0.002s
I20260812 19:07:45.856318 30670 rpcz_store.cc:276] 0812 19:07:44.564814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.590349 (+1025535us) service_pool.cc:227] Handling call
0812 19:07:45.626106 (+ 35757us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.635284 (+ 9178us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:45.735328 (+100044us) log.cc:842] Serialized 14 byte log entry
0812 19:07:45.809064 (+ 73736us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.812912 (+ 3848us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:45.817134 (+ 4222us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.827269 (+ 10135us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:45.836920 (+ 9651us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":38615,"prepare.queue_time_us":97288,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":10518,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3293,"thread_start_us":108505,"threads_started":2,"wal-append.queue_time_us":103957}
I20260812 19:07:45.892313 29574 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:45.886643 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 94802.9us
W20260812 19:07:45.873330 745 consensus_meta.cc:287] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.534s user 0.006s sys 0.005s
I20260812 19:07:45.901360 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.880152 29453 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:45.895710 29446 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:45.899077 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.903021 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:45.903072 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 236667us
I20260812 19:07:45.904122 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:45.905642 29569 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260812 19:07:45.875012 1262 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:45.910147 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.912977 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.914887 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:45.915464 30686 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 2: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:45.901151 1083 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.911670 1288 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.911629 1311 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:45.930706 29574 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:45.920819 1328 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:45.934336 936 raft_consensus.cc:3032] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:45.934768 936 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:45.881783 1266 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.898217 924 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.937633 29527 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.944643 29527 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.945721 29527 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:45.946022 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:37750 (request call id 872) took 1091 ms. Trace:
I20260812 19:07:45.946264 777 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:45.946460 29337 rpcz_store.cc:276] 0812 19:07:44.861900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.103146 (+241246us) service_pool.cc:227] Handling call
0812 19:07:45.945294 (+842148us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54003903d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6ed53c6e 00007f9c68456807 00007f9c68456792 00007f9c684566c1 00007f9c6845663c 00007f9c684558f2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:45.945940 (+ 646us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1781676928}
I20260812 19:07:45.946884 777 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.935575 936 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.910841 745 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:45.907672 1209 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.906711 825 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:45.947996 1208 consensus_meta.cc:287] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.685s user 0.011s sys 0.000s
W20260812 19:07:45.949260 1329 log.cc:924] Time spent T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260812 19:07:45.949481 1093 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.950732 29580 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:45.943470 1311 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:45.949245 1152 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.951090 745 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:45.950621 1209 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.952303 825 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:45.949684 922 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:45.955104 1328 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:45.947959 29338 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:45.958184 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:45.970566 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 57609.2us
I20260812 19:07:45.984529 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:45.951817 1257 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:45.988981 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:45.987412 745 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:45.951884 936 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.050463 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:46.053412 29580 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:45.988508 1166 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:46.053296 29399 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.049044 1328 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:46.060539 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.071926 1257 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:46.072075 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.057268 1166 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:46.053550 1338 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:46.073652 1338 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:46.073805 1166 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.074643 1338 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.074874 1330 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:46.075287 1330 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.075611 1330 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.064344 1323 consensus_queue.cc:1046] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:46.066901 924 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:46.068820 30686 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:46.070367 1266 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:46.074167 1328 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:46.062565 1327 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260812 19:07:46.080641 1209 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:46.085878 1328 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.086350 1328 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.088341 30685 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:46.077632 1338 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:46.073210 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:46.050208 1134 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:46.078608 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.091787 29580 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.092118 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:46.092978 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.094548 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.095418 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:46.096449 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.097765 1134 raft_consensus.cc:686] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.099370 30676 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260812 19:07:46.102701 29578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2073) took 1684 ms (client timeout 2003 ms). Trace:
W20260812 19:07:46.103206 29578 rpcz_store.cc:269] 0812 19:07:44.418471 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.471326 (+ 52855us) service_pool.cc:227] Handling call
0812 19:07:46.102493 (+1631167us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b5400178754. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:46.102635 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":106557312}
I20260812 19:07:46.104592 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2064) took 1703 ms. Trace:
I20260812 19:07:46.104993 29575 rpcz_store.cc:276] 0812 19:07:44.401395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.665558 (+264163us) service_pool.cc:227] Handling call
0812 19:07:44.665857 (+ 299us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:44.727928 (+ 62071us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.104344 (+1376416us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b54002008d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.104528 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":18887040}
I20260812 19:07:46.079682 1166 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:46.093920 1000 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.102175 1311 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:46.106003 1288 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:46.106171 1070 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:46.101059 924 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:46.123718 29580 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:46.127576 30623 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:46.128495 1205 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:46.128984 1205 raft_consensus.cc:686] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.109373 1266 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260812 19:07:46.118939 1311 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.098806 1134 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.126672 1070 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.139935 1288 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:46.144299 924 consensus_queue.cc:1046] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260812 19:07:46.152122 924 consensus_queue.cc:1225] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260812 19:07:46.152235 1268 log.cc:924] Time spent T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.494s user 0.001s sys 0.001s
I20260812 19:07:46.146229 1311 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.149367 1288 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.162690 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2075) took 1743 ms. Trace:
I20260812 19:07:46.149514 1070 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.163192 29579 rpcz_store.cc:276] 0812 19:07:44.418966 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.704262 (+285296us) service_pool.cc:227] Handling call
0812 19:07:44.704542 (+ 280us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:44.714554 (+ 10012us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:44.714583 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:44.715033 (+ 450us) log.cc:842] Serialized 26 byte log entry
0812 19:07:46.108442 (+1393409us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b18001f0600. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ed4345 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb 00007f9c69a95063 00007f9c69a95411 00007f9c69de25cf
0812 19:07:46.142885 (+ 34443us) raft_consensus.cc:1631] Marking committed up to 1
0812 19:07:46.142920 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.142981 (+ 61us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:46.159841 (+ 16860us) raft_consensus.cc:1671] finished
0812 19:07:46.159858 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.162628 (+ 2770us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9516,"raft.queue_time_us":9636,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":60276,"spinlock_wait_cycles":38499712,"thread_start_us":34249,"threads_started":1,"wal-append.queue_time_us":1376265}
I20260812 19:07:46.129856 1205 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.174533 1172 raft_consensus.cc:3032] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:46.164758 1288 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.164358 29574 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
W20260812 19:07:46.180717 30675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 774) took 3304 ms (client timeout 1546 ms). Trace:
W20260812 19:07:46.181231 30675 rpcz_store.cc:269] 0812 19:07:42.875881 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:42.881251 (+ 5370us) service_pool.cc:227] Handling call
0812 19:07:46.180480 (+3299229us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54002fa654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:46.180632 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4796227840}
I20260812 19:07:46.179078 1172 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.184314 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 572) took 3687 ms. Trace:
I20260812 19:07:46.184690 29577 rpcz_store.cc:276] 0812 19:07:42.497240 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:43.590574 (+1093334us) service_pool.cc:227] Handling call
0812 19:07:43.670527 (+ 79953us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.266126 (+1595599us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.184080 (+917954us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400037ad4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.184258 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3547682048}
I20260812 19:07:46.192801 1262 consensus_queue.cc:1046] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260812 19:07:46.199132 1262 raft_consensus.cc:3032] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:46.199666 1262 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.200793 1338 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.202468 1328 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.203478 1162 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:46.204151 1338 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.183145 1172 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.183209 1081 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.218611 1081 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:46.218381 1328 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.222003 1070 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.222813 1311 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:46.228832 1342 log.cc:924] Time spent T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.076s user 0.001s sys 0.000s
I20260812 19:07:46.232816 1311 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.200872 1262 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.234215 1262 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.237342 1288 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.238588 1330 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.239521 1288 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.240679 1330 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.274178 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1918) took 2116 ms. Trace:
W20260812 19:07:46.274924 29528 leader_election.cc:334] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.546s (SENT)
I20260812 19:07:46.276002 29528 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:46.281562 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.283113 1266 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:46.311769 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 750) took 1898 ms. Trace:
I20260812 19:07:46.312356 30684 rpcz_store.cc:276] 0812 19:07:44.412791 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.565288 (+1152497us) service_pool.cc:227] Handling call
0812 19:07:46.092019 (+526731us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:46.092189 (+ 170us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:46.092213 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:46.092253 (+ 40us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:46.092287 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.092336 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.311238 (+218902us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54002fb550. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:46.311463 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1210604800}
I20260812 19:07:46.310374 29579 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:46.314919 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2063) took 3396 ms. Trace:
I20260812 19:07:46.315400 29576 rpcz_store.cc:276] 0812 19:07:42.918109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.579408 (+1661299us) service_pool.cc:227] Handling call
0812 19:07:44.579735 (+ 327us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.384576 (+804841us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.314382 (+929806us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54000375d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.314716 (+ 334us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1837987712}
I20260812 19:07:46.317726 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 749) took 1907 ms. Trace:
I20260812 19:07:46.318185 30671 rpcz_store.cc:276] 0812 19:07:44.410080 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.371968 (+961888us) service_pool.cc:227] Handling call
0812 19:07:45.372276 (+ 308us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.401635 (+ 29359us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.317121 (+915486us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b540070d854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.317469 (+ 348us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":64583552}
I20260812 19:07:46.275414 29445 rpcz_store.cc:276] 0812 19:07:44.157987 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.081865 (+923878us) service_pool.cc:227] Handling call
0812 19:07:45.082331 (+ 466us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:45.082729 (+ 398us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.082756 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:45.095755 (+ 12999us) log.cc:842] Serialized 26 byte log entry
0812 19:07:45.436458 (+340703us) raft_consensus.cc:1631] Marking committed up to 1
0812 19:07:45.436505 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:45.436583 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:46.151666 (+715083us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:46.272897 (+121231us) raft_consensus.cc:1671] finished
0812 19:07:46.273002 (+ 105us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.273782 (+ 780us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":22763,"spinlock_wait_cycles":9549568,"thread_start_us":292719,"threads_started":1,"wal-append.queue_time_us":519686}
W20260812 19:07:46.324240 30685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 751) took 1908 ms (client timeout 2008 ms). Trace:
W20260812 19:07:46.324733 30685 rpcz_store.cc:269] 0812 19:07:44.416098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:44.545077 (+128979us) service_pool.cc:227] Handling call
0812 19:07:45.861041 (+1315964us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:46.323758 (+462717us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400465854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:46.323986 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":533287552}
W20260812 19:07:46.291124 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 151308us
I20260812 19:07:46.325632 30623 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:46.330854 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 585) took 1254 ms. Trace:
I20260812 19:07:46.344000 29572 rpcz_store.cc:276] 0812 19:07:45.076715 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.338447 (+261732us) service_pool.cc:227] Handling call
0812 19:07:46.102677 (+764230us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:46.102885 (+ 208us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:46.102910 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:46.102955 (+ 45us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:46.102992 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.103046 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.330332 (+227286us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b5400178750. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:46.330580 (+ 248us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1756788992}
I20260812 19:07:46.336452 29578 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:46.360942 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:46.371932 709 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:46.372606 709 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.373062 709 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.373329 1330 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:46.373863 1330 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:46.374779 1330 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.380328 702 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.386365 30676 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:46.386863 1070 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.391757 709 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.394775 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 753) took 1765 ms. Trace:
I20260812 19:07:46.395192 30681 rpcz_store.cc:276] 0812 19:07:44.629088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.622639 (+993551us) service_pool.cc:227] Handling call
0812 19:07:45.622919 (+ 280us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.646767 (+ 23848us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.394264 (+747497us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400480dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.394602 (+ 338us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":53274496}
I20260812 19:07:46.395574 709 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:46.399803 708 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.422264 29332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 696) took 1006 ms. Trace:
I20260812 19:07:46.422832 29332 rpcz_store.cc:276] 0812 19:07:45.387801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.400614 (+ 12813us) service_pool.cc:227] Handling call
0812 19:07:45.409133 (+ 8519us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.413483 (+ 4350us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:45.413517 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:45.413559 (+ 42us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:45.413597 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.393567 (+979970us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b5400356454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.393642 (+ 75us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.393876 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9512832}
W20260812 19:07:46.422276 1354 log.cc:924] Time spent T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.059s user 0.002s sys 0.000s
I20260812 19:07:46.427484 29530 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:46.431294 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 81121.3us
I20260812 19:07:46.431706 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:46.377763 1269 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.776s user 0.000s sys 0.002s
I20260812 19:07:46.433740 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:46.460786 29445 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.464888 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.466593 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.467617 29342 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:46.468242 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.468437 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.469533 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.436735 1292 raft_consensus.cc:3032] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:46.482316 1292 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:46.438890 29528 leader_election.cc:334] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.603s (SENT)
W20260812 19:07:46.440452 1299 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.661s user 0.002s sys 0.000s
I20260812 19:07:46.442443 29338 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:46.445358 1155 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 19:07:46.446295 29458 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:46.447966 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 692) took 1275 ms. Trace:
I20260812 19:07:46.448916 1152 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:46.450220 1355 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.078s user 0.001s sys 0.000s
W20260812 19:07:46.434856 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 833768us
I20260812 19:07:46.470534 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.497998 29528 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:46.434733 1293 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:46.479547 29332 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:07:46.480329 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.466581 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:46.496726 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 107027us
I20260812 19:07:46.475831 1172 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.502843 1152 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.505636 29445 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:46.505909 1152 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.506809 1292 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.501116 1293 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:46.509452 1293 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.510376 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:46.511255 1293 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:46.511663 1000 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 19:07:46.512399 30625 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.744s (SENT) [suppressed 33 similar messages]
I20260812 19:07:46.513732 1330 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.516186 1330 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.519297 709 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:46.519989 709 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.521183 1155 raft_consensus.cc:686] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:46.522682 1155 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.495085 29326 rpcz_store.cc:276] 0812 19:07:45.172516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.172681 (+ 165us) service_pool.cc:227] Handling call
0812 19:07:45.173017 (+ 336us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:45.743324 (+570307us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.447359 (+704035us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54002319d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:46.447712 (+ 353us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1310261120}
I20260812 19:07:46.511212 30681 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:46.535167 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:46.536093 30676 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:46.544636 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.545369 29578 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.550251 1140 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260812 19:07:46.550732 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 111959us
W20260812 19:07:46.553265 29528 leader_election.cc:334] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.754s (SENT)
I20260812 19:07:46.553849 29528 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.555024 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.555789 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.556959 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.557219 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.561293 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.562703 825 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:46.557332 1000 consensus_queue.cc:1046] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260812 19:07:46.565667 1287 raft_consensus.cc:3032] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:46.531313 708 consensus_queue.cc:1046] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260812 19:07:46.566067 1287 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:46.566486 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:46.567274 1287 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.568310 1302 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:46.569137 1287 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.569008 1302 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.570140 29579 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:46.574565 708 consensus_queue.cc:1225] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260812 19:07:46.575145 708 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:46.575510 708 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:46.575688 1292 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.576751 708 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.586393 1279 consensus_queue.cc:1046] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:46.587879 1281 raft_consensus.cc:3032] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:46.588317 1281 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.589359 1281 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.590965 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.591946 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.592234 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.592618 30671 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:46.593051 29567 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:46.592916 1281 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.593677 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.595801 30676 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:46.597898 30670 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:46.586288 1295 consensus_queue.cc:1046] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260812 19:07:46.600832 1295 raft_consensus.cc:3032] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:46.601261 1295 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.582916 708 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.607638 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.620535 1160 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:46.625139 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:46.628264 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 45140.7us
W20260812 19:07:46.633183 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 70394.2us
I20260812 19:07:46.632853 1330 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:46.639719 29579 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:46.641101 29578 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:46.648115 30676 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:46.661924 30623 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:46.664466 29527 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:46.665306 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 68528.5us
I20260812 19:07:46.665997 29527 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:46.666927 1208 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:46.667295 1208 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.667354 1152 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:46.667869 1152 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.668280 1152 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.640403 1295 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.671236 1295 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:46.674009 29528 leader_election.cc:334] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.920s (SENT)
I20260812 19:07:46.680408 30685 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:46.667668 1208 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.678577 1081 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:46.676229 1152 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.680848 29528 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.657166 1330 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:46.692190 702 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:46.701016 1160 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:46.701251 30671 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:46.690312 1081 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.702831 1081 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.703140 1152 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.702325 1330 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:46.707664 928 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.003s
I20260812 19:07:46.709403 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:46.712860 29342 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:46.719161 29282 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:46.721230 29282 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:46.722275 1061 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:46.722930 1061 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.723881 709 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:46.724184 1061 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:46.731536 1208 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:46.739172 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 47066.1us
W20260812 19:07:46.738794 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 150044us
I20260812 19:07:46.753532 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.754328 30685 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:46.743336 702 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:46.747450 1281 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260812 19:07:46.766604 1323 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260812 19:07:46.768714 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.772936 29282 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:46.776203 29337 raft_consensus.cc:2379] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:46.751981 1140 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:07:46.787935 1294 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:46.788578 1294 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:46.795393 30625 leader_election.cc:334] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.752s (SENT)
I20260812 19:07:46.795307 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.795945 30625 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.790658 1302 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:46.810909 29567 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:46.795233 1140 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.813614 29341 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:46.814141 706 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:46.814767 1281 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:46.813606 29579 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:46.812804 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 67858.1us
I20260812 19:07:46.818097 1025 consensus_queue.cc:1046] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:46.814165 1270 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260812 19:07:46.772096 702 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.832912 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.835300 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:46.837852 29528 leader_election.cc:334] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.978s (SENT)
I20260812 19:07:46.838300 1025 raft_consensus.cc:3032] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:46.838747 1025 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.839748 1025 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.841432 1025 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:46.838416 29528 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:46.834455 30685 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:46.844326 29578 raft_consensus.cc:2354] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
W20260812 19:07:46.847652 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 57324.4us
I20260812 19:07:46.849735 29527 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:46.854756 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.855386 30671 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:46.835147 1208 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:46.850625 1292 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:46.846024 706 raft_consensus.cc:686] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.858922 1140 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:46.844072 928 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260812 19:07:46.867082 29578 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260812 19:07:46.867484 29578 pending_rounds.cc:76] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting all ops after (but not including) 7
W20260812 19:07:46.863926 30625 leader_election.cc:334] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.814s (SENT)
I20260812 19:07:46.867975 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:46.867722 29578 pending_rounds.cc:97] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260812 19:07:46.871003 29578 raft_consensus.cc:2866] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:46.866474 708 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:46.861241 1160 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:46.874161 1160 raft_consensus.cc:686] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:46.874380 1279 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:46.874667 1270 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:46.873664 29176 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 387) took 5618 ms. Trace:
I20260812 19:07:46.878966 29282 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:46.862473 1292 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:46.882524 29580 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260812 19:07:46.883630 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:46.891588 30676 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:46.884970 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:46.886315 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 48544.7us
I20260812 19:07:46.878438 29176 rpcz_store.cc:276] 0812 19:07:41.247757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:41.259989 (+ 12232us) service_pool.cc:227] Handling call
0812 19:07:46.861109 (+5601120us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:45.327796 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:45.341220 (+ 13424us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:45.341261 (+ 41us) write_op.cc:435] Acquired schema lock
0812 19:07:45.341280 (+ 19us) tablet.cc:660] Decoding operations
0812 19:07:45.342036 (+ 756us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:45.342117 (+ 81us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:45.342143 (+ 26us) tablet.cc:683] Acquiring locks for 5 operations
0812 19:07:45.342784 (+ 641us) tablet.cc:699] Row locks acquired
0812 19:07:45.342809 (+ 25us) write_op.cc:260] PREPARE: finished
0812 19:07:45.342975 (+ 166us) write_op.cc:270] Start()
0812 19:07:45.343109 (+ 134us) write_op.cc:276] Timestamp: P: 1786561665342923 usec, L: 0
0812 19:07:45.343154 (+ 45us) op_driver.cc:331] REPLICATION: starting
0812 19:07:45.462673 (+119519us) log.cc:842] Serialized 2137 byte log entry
0812 19:07:45.740184 (+277511us) op_driver.cc:444] REPLICATION: finished
0812 19:07:45.868228 (+128044us) write_op.cc:301] APPLY: starting
0812 19:07:45.868315 (+ 87us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:45.868478 (+ 163us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:45.871324 (+ 2846us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:45.913771 (+ 42447us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:45.931531 (+ 17760us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0812 19:07:45.934939 (+ 3408us) write_op.cc:312] APPLY: finished
0812 19:07:45.935197 (+ 258us) log.cc:842] Serialized 47 byte log entry
0812 19:07:45.935363 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:45.935469 (+ 106us) write_op.cc:454] Released schema lock
0812 19:07:45.935512 (+ 43us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:45.938251 (+ 2739us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:45.940211 (+ 1960us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":115896,"apply.run_cpu_time_us":3767,"apply.run_wall_time_us":72758,"num_ops":5,"prepare.queue_time_us":1222122,"prepare.run_cpu_time_us":17892,"prepare.run_wall_time_us":252775,"raft.queue_time_us":34017,"raft.run_cpu_time_us":19690,"raft.run_wall_time_us":68015,"replication_time_us":396933,"spinlock_wait_cycles":1649024,"thread_start_us":1115275,"threads_started":4,"wal-append.queue_time_us":151179}]]}
I20260812 19:07:46.894646 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2267) took 1191 ms. Trace:
I20260812 19:07:46.893698 1292 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:46.902956 30671 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:46.891981 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:46.909523 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.911324 30685 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:46.915730 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:46.919449 1134 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260812 19:07:46.896150 29339 rpcz_store.cc:276] 0812 19:07:45.614647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:45.614804 (+ 157us) service_pool.cc:227] Handling call
0812 19:07:45.625652 (+ 10848us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:46.594054 (+968402us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:46.596947 (+ 2893us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:46.612720 (+ 15773us) log.cc:842] Serialized 26 byte log entry
0812 19:07:46.753276 (+140556us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:46.753313 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:46.759737 (+ 6424us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:46.803327 (+ 43590us) raft_consensus.cc:1671] finished
0812 19:07:46.803346 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:46.806027 (+ 2681us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":811453,"raft.queue_time_us":819173,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":236,"thread_start_us":198526,"threads_started":2,"wal-append.queue_time_us":186428}
W20260812 19:07:46.928299 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 55279.7us
I20260812 19:07:46.930341 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.934573 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:46.938345 1362 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:46.939905 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.904018 709 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260812 19:07:46.943233 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.883625 706 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.946354 1176 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:46.947067 1176 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.886729 1160 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:46.957437 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:46.958228 29331 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:46.961239 29580 raft_consensus.cc:2354] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7.
I20260812 19:07:46.962276 29341 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:46.963064 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:46.941793 1362 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:46.941372 1294 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:46.930138 708 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260812 19:07:46.970778 29332 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:46.969823 1206 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:46.979629 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:46.981218 29580 raft_consensus.cc:2354] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7.
I20260812 19:07:46.969997 1208 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:46.984490 29341 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:46.975136 1206 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:46.966980 1061 consensus_queue.cc:1046] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260812 19:07:47.003341 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.006594 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.011873 29567 raft_consensus.cc:2354] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:47.017656 30671 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:47.017938 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.002712 1206 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:47.021003 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:47.025516 29342 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:46.994201 1208 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.021251 709 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:47.021658 29528 leader_election.cc:334] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.178s (SENT)
I20260812 19:07:47.002516 1176 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:47.024057 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.020756 1362 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.012738 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.033170 29528 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:47.036832 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:47.039180 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.038239 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:47.042725 29339 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:47.043889 30681 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:47.051070 29445 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260812 19:07:47.052240 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.053063 29458 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260812 19:07:47.061685 1378 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260812 19:07:47.054826 29332 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:47.060877 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:47.054826 30681 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:47.065285 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.065615 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.066210 29576 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:47.066910 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.018610 1330 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.069787 29577 raft_consensus.cc:2396] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 2 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 19:07:47.066371 29578 raft_consensus.cc:2354] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9.
I20260812 19:07:47.052574 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.067626 29527 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:47.079577 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.080616 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.081260 29579 raft_consensus.cc:2354] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9.
I20260812 19:07:47.042420 708 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:47.075531 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.083113 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.071707 30670 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 8: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:47.091845 30684 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:47.088729 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.092623 29567 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:47.090112 1155 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:47.086265 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.096679 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.097381 29451 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:47.075958 1176 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:47.053607 709 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.112402 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.045876 1257 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
W20260812 19:07:47.111863 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 94108.9us
I20260812 19:07:47.097548 1259 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:47.114389 29458 raft_consensus.cc:2379] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.114821 29453 raft_consensus.cc:2379] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.114953 29445 raft_consensus.cc:2396] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:47.117564 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.070226 1081 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.102571 1208 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:47.108453 1155 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:47.132856 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.133422 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:47.133759 29459 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.136555 998 consensus_queue.cc:1046] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:47.137291 998 consensus_queue.cc:1225] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260812 19:07:47.133011 763 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:47.138731 1152 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:47.138898 763 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:47.139348 709 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:47.116598 1176 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:47.140400 1330 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:47.139930 763 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.139971 30686 raft_consensus.cc:1203] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Deduplicated request from leader. Original: 4.7->[6.8-6.8] Dedup: 6.8->[]
I20260812 19:07:47.142067 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:47.143056 29451 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:47.142118 30686 raft_consensus.cc:1226] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260812 19:07:47.149807 30671 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:47.155287 30676 raft_consensus.cc:2379] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:47.139297 1152 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:47.141390 763 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:47.164036 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:47.164918 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:47.156172 1371 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
I20260812 19:07:47.152215 708 raft_consensus.cc:3032] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:47.151584 1259 raft_consensus.cc:686] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:47.173833 30676 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:47.174228 29451 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
W20260812 19:07:47.176064 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 53095.8us
I20260812 19:07:47.179167 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.179741 1081 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:47.179972 29445 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.180616 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.181334 29458 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:47.181979 1141 consensus_queue.cc:1046] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:47.183180 1141 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:47.185048 1061 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:47.185431 1061 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:47.177665 29451 pending_rounds.cc:76] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 7
I20260812 19:07:47.186188 29451 pending_rounds.cc:97] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260812 19:07:47.186472 29451 raft_consensus.cc:2866] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:47.186362 1061 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:47.187729 1061 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:47.188076 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.190629 1259 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.191098 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.191385 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.191857 29458 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.192452 29445 raft_consensus.cc:2379] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.192363 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.194466 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:47.199559 1292 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.200804 30685 raft_consensus.cc:2354] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:47.191895 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:47.201669 30676 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:47.141345 1155 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.204900 1206 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.140710 1176 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.206897 706 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:47.211807 29452 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:07:47.215706 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:47.217288 1083 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:47.218888 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:47.219144 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.219519 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.220180 29451 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:47.221369 29445 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:47.226595 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.227185 29458 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:47.164052 1160 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:47.174429 708 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:47.201723 1292 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:47.261801 29459 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260812 19:07:47.290271 29445 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:47.233608 1257 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260812 19:07:47.295614 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 79990.2us
W20260812 19:07:47.298600 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 165295us
W20260812 19:07:47.303591 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 90043.8us
I20260812 19:07:47.297858 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.304690 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:47.305176 29459 raft_consensus.cc:2396] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:47.307768 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.309131 29452 raft_consensus.cc:2379] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.317010 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:47.317775 29332 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:47.323915 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.315908 708 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:47.326818 30685 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260812 19:07:47.330295 29331 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:47.332703 29577 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:07:47.289124 1160 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.338860 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.347694 30686 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260812 19:07:47.326977 1083 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:47.305711 1152 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:47.357182 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.357726 29567 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:47.362135 1395 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:47.362545 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.363400 29457 raft_consensus.cc:2379] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:47.372249 30686 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:47.354765 1083 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:47.379319 1379 log.cc:924] Time spent T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.248s user 0.001s sys 0.001s
I20260812 19:07:47.381140 29572 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260812 19:07:47.383543 29457 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:47.385135 29459 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:47.385715 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:47.357012 1206 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:47.373628 708 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:47.391875 1083 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:47.408234 1206 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:47.415576 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.416864 29459 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:47.418381 29458 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:47.429774 29459 raft_consensus.cc:1226] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:47.430972 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:47.438537 29457 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:47.441011 29457 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260812 19:07:47.440701 1140 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:47.440915 1330 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:47.441632 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 105290us
I20260812 19:07:47.441752 29457 pending_rounds.cc:76] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 4
I20260812 19:07:47.464493 29457 pending_rounds.cc:97] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260812 19:07:47.464809 29457 raft_consensus.cc:2866] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:47.462249 1362 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260812 19:07:47.465606 1134 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.468868 1134 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.470990 1134 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:47.472445 29458 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:07:47.431514 1393 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260812 19:07:47.474426 1160 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:47.476565 29339 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:47.479569 1270 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:47.481623 1061 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:47.482592 1193 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:47.484828 1293 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.486367 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.487172 29459 raft_consensus.cc:2379] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.488051 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.488761 29459 raft_consensus.cc:2354] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:47.489544 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.492921 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:47.502867 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:47.441418 1176 consensus_queue.cc:1046] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260812 19:07:47.485463 1293 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:47.511584 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.510702 30684 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:47.514506 30676 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:47.516664 30685 raft_consensus.cc:2379] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 2: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:47.536823 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:47.537870 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.556828 29459 raft_consensus.cc:2354] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7.
I20260812 19:07:47.539130 29457 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:47.490672 1330 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:47.589917 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.598593 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.586958 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:47.578922 1393 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:47.609370 29459 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:47.617177 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:47.562357 1206 consensus_queue.cc:1046] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260812 19:07:47.634516 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:47.642529 29451 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:47.649194 29458 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:47.660887 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.581846 1176 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260812 19:07:47.666685 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.610407 1172 consensus_queue.cc:1046] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260812 19:07:47.681012 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.681769 29451 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:47.690224 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.672415 29572 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:47.745445 1081 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.745927 30623 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:47.746125 1081 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:47.746732 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 134117us
I20260812 19:07:47.748371 1081 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:47.754371 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.764333 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.765677 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.766796 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.767485 29459 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:47.753711 1081 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:47.783654 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 209573us
I20260812 19:07:47.804137 1081 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:47.804491 1403 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.268s user 0.000s sys 0.001s
I20260812 19:07:47.804647 1081 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:47.768529 1326 log.cc:924] Time spent T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.095s user 0.000s sys 0.000s
I20260812 19:07:47.776341 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:47.816876 29451 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:47.818182 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:47.818955 29451 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:47.823858 29567 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:47.829284 29579 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:47.832427 1323 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:47.713033 1257 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260812 19:07:47.814607 1386 log.cc:924] Time spent T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
W20260812 19:07:48.005148 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 111437us
I20260812 19:07:48.228456 29576 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260812 19:07:48.235081 30625 leader_election.cc:334] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.164s (SENT)
W20260812 19:07:48.257009 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 439563us
I20260812 19:07:47.893745 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:48.466202 29459 raft_consensus.cc:2362] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7. Re-sending same reply.
W20260812 19:07:48.249789 29453 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s user 0.001s sys 0.014s
I20260812 19:07:48.391830 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:48.624552 29451 consensus_meta.cc:287] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.805s user 0.008s sys 0.001s
I20260812 19:07:48.957882 29452 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
W20260812 19:07:48.810761 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.785s user 0.004s sys 0.008s
W20260812 19:07:48.959106 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.933s user 0.004s sys 0.011s
I20260812 19:07:48.518994 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:48.885111 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1956) took 1593 ms. Trace:
W20260812 19:07:48.677919 29446 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.905s user 0.005s sys 0.000s
W20260812 19:07:47.912079 29528 leader_election.cc:334] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.847s (SENT)
I20260812 19:07:48.963514 29446 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:48.502822 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:48.464864 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:48.964341 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:48.968380 29341 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:48.958305 29451 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:48.733480 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2112) took 1249 ms. Trace:
I20260812 19:07:48.965029 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2382) took 1477 ms. Trace:
W20260812 19:07:48.969974 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 1.07643e+06us
I20260812 19:07:48.025770 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:48.971149 29457 raft_consensus.cc:2396] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260812 19:07:48.528925 29572 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s user 0.002s sys 0.025s
W20260812 19:07:48.971603 29572 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.204s user 0.005s sys 0.035s
I20260812 19:07:48.967546 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:48.964826 29453 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.307s user 0.008s sys 0.031s
I20260812 19:07:48.962400 29445 rpcz_store.cc:276] 0812 19:07:47.291890 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.292137 (+ 247us) service_pool.cc:227] Handling call
0812 19:07:47.302378 (+ 10241us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:47.319370 (+ 16992us) raft_consensus.cc:1520] Early marking committed up to index 2
0812 19:07:47.690819 (+371449us) log.cc:842] Serialized 14 byte log entry
0812 19:07:48.866114 (+1175295us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:48.866172 (+ 58us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:48.884883 (+ 18711us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:48.884952 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:48.885052 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16054,"prepare.queue_time_us":49118,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"thread_start_us":769309,"threads_started":3,"wal-append.queue_time_us":821051}
W20260812 19:07:48.870716 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 353301us
W20260812 19:07:48.865767 29283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.000s sys 0.007s
W20260812 19:07:48.978717 29283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.616s user 0.000s sys 0.008s
I20260812 19:07:48.971288 29456 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:48.970630 29578 rpcz_store.cc:276] 0812 19:07:47.484421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.810079 (+325658us) service_pool.cc:227] Handling call
0812 19:07:48.732956 (+922877us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:48.733128 (+ 172us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:48.733153 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:48.733191 (+ 38us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:48.733227 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:48.733280 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:48.733426 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":65024}
W20260812 19:07:48.458132 1432 log.cc:924] Time spent T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.447s user 0.000s sys 0.001s
W20260812 19:07:48.674099 1401 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.807s user 0.001s sys 0.000s
W20260812 19:07:48.069146 1405 log.cc:924] Time spent T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.169s user 0.000s sys 0.000s
W20260812 19:07:48.457391 1444 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.431s user 0.000s sys 0.000s
W20260812 19:07:48.982671 29446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2370) took 1483 ms (client timeout 1941 ms). Trace:
W20260812 19:07:48.983741 29451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2397) took 1155 ms (client timeout 1519 ms). Trace:
I20260812 19:07:48.984061 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2376) took 1498 ms. Trace:
I20260812 19:07:48.992336 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 648) took 1167 ms. Trace:
I20260812 19:07:48.519845 709 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:48.468180 763 consensus_meta.cc:287] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 1.291s user 0.008s sys 0.004s
I20260812 19:07:48.970803 29452 rpcz_store.cc:276] 0812 19:07:47.487253 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:48.225622 (+738369us) service_pool.cc:227] Handling call
0812 19:07:48.964970 (+739348us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:48.667184 1083 consensus_meta.cc:287] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.221s user 0.007s sys 0.003s
W20260812 19:07:48.234640 1437 log.cc:924] Time spent T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.415s user 0.000s sys 0.001s
W20260812 19:07:48.069833 1423 log.cc:924] Time spent T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.245s user 0.000s sys 0.000s
W20260812 19:07:48.265065 1409 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.664s user 0.001s sys 0.001s
I20260812 19:07:48.391376 708 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.826189 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:48.663455 1081 consensus_meta.cc:287] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.858s user 0.010s sys 0.000s
W20260812 19:07:48.985265 30679 consensus_meta.cc:287] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.519s user 0.008s sys 0.004s
W20260812 19:07:48.992393 29451 rpcz_store.cc:269] 0812 19:07:47.815757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.817907 (+ 2150us) service_pool.cc:227] Handling call
0812 19:07:48.970576 (+1152669us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:48.065490 1443 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:48.984273 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2372) took 1496 ms. Trace:
I20260812 19:07:48.463054 1025 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:49.011360 29456 rpcz_store.cc:276] 0812 19:07:47.484643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.912322 (+427679us) service_pool.cc:227] Handling call
0812 19:07:48.981154 (+1068832us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:48.631446 1462 log.cc:924] Time spent T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260812 19:07:48.992595 29457 rpcz_store.cc:276] 0812 19:07:47.485849 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.804013 (+318164us) service_pool.cc:227] Handling call
0812 19:07:48.984001 (+1179988us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.013927 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.014081 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:48.993594 29567 rpcz_store.cc:276] 0812 19:07:47.824488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.902526 (+ 78038us) service_pool.cc:227] Handling call
0812 19:07:48.991619 (+1089093us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:48.991827 (+ 208us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:48.992008 (+ 181us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:48.992033 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:48.992073 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:48.992109 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:48.992161 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:48.992279 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":607500544}
I20260812 19:07:48.068372 1440 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:47.900755 1293 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:48.991797 29446 rpcz_store.cc:269] 0812 19:07:47.484037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.484483 (+ 446us) service_pool.cc:227] Handling call
0812 19:07:48.967635 (+1483152us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:48.573239 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 769) took 1217 ms. Trace:
I20260812 19:07:48.925912 1259 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:48.503010 1330 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:48.998847 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:48.455945 1172 consensus_queue.cc:1046] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.545s
I20260812 19:07:48.993460 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:49.094868 1259 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:49.085003 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:49.094190 29445 raft_consensus.cc:2362] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3. Re-sending same reply.
I20260812 19:07:48.836545 1140 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.164s
W20260812 19:07:49.064414 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 72460us
I20260812 19:07:49.012472 1081 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.002043 1193 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.085073 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.005746 1453 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:49.005010 763 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:49.085035 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.085316 1481 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:49.096711 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2396) took 1356 ms. Trace:
I20260812 19:07:49.089128 29457 raft_consensus.cc:2396] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:07:49.096616 29446 raft_consensus.cc:2362] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7. Re-sending same reply.
I20260812 19:07:49.098716 29459 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:49.062682 29451 raft_consensus.cc:2362] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4. Re-sending same reply.
I20260812 19:07:49.012631 1025 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:49.007043 1206 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:49.000542 709 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.106978 30686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 775) took 1730 ms. Trace:
I20260812 19:07:49.060993 1434 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.106889 29342 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2280) took 1741 ms. Trace:
I20260812 19:07:49.014446 1449 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:49.114382 29445 rpcz_store.cc:276] 0812 19:07:47.740019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:48.983826 (+1243807us) service_pool.cc:227] Handling call
0812 19:07:49.096650 (+112824us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.113533 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 832) took 1966 ms. Trace:
I20260812 19:07:49.114902 29456 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:49.115751 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2374) took 1630 ms. Trace:
I20260812 19:07:49.107972 1470 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:49.127843 30686 rpcz_store.cc:276] 0812 19:07:47.376167 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.393500 (+ 17333us) service_pool.cc:227] Handling call
0812 19:07:47.409363 (+ 15863us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.409726 (+ 363us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.409751 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.415139 (+ 5388us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.515751 (+100612us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:47.515792 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.515864 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.106772 (+1590908us) raft_consensus.cc:1671] finished
0812 19:07:49.106792 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.106912 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3520,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2131,"thread_start_us":84319,"threads_started":1,"wal-append.queue_time_us":230956}
I20260812 19:07:49.141518 29459 rpcz_store.cc:276] 0812 19:07:47.485312 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:48.517383 (+1032071us) service_pool.cc:227] Handling call
0812 19:07:49.115693 (+598310us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.016635 29574 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:49.116182 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2398) took 1203 ms. Trace:
I20260812 19:07:49.119233 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2384) took 1631 ms. Trace:
I20260812 19:07:49.102864 1330 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.003458 1083 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.136626 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.134882 29342 rpcz_store.cc:276] 0812 19:07:47.365677 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.365840 (+ 163us) service_pool.cc:227] Handling call
0812 19:07:47.366219 (+ 379us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.366596 (+ 377us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.366622 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.367093 (+ 471us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.558974 (+191881us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:47.559054 (+ 80us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.559122 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.106690 (+1547568us) raft_consensus.cc:1671] finished
0812 19:07:49.106711 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.106830 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3476,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":12439,"thread_start_us":191520,"threads_started":1,"wal-append.queue_time_us":343730}
W20260812 19:07:49.118253 1385 log.cc:924] Time spent T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 1.377s user 0.005s sys 0.014s
I20260812 19:07:49.109248 1295 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:49.139318 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.154568 29445 raft_consensus.cc:2396] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 19:07:49.010996 30679 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:49.011883 708 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.019754 1443 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.114677 1482 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260812 19:07:49.114138 1453 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.145695 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2383) took 1658 ms. Trace:
I20260812 19:07:49.145887 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.119331 763 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:49.147261 29451 rpcz_store.cc:276] 0812 19:07:47.912958 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.013796 (+1100838us) service_pool.cc:227] Handling call
0812 19:07:49.116125 (+102329us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.147854 29452 rpcz_store.cc:276] 0812 19:07:47.487797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.060924 (+1573127us) service_pool.cc:227] Handling call
0812 19:07:49.119065 (+ 58141us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.148039 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.151018 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2281) took 1781 ms. Trace:
I20260812 19:07:49.152376 29446 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:49.066107 1440 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:49.157851 30679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 776) took 1749 ms (client timeout 1653 ms). Trace:
I20260812 19:07:49.161912 29575 raft_consensus.cc:2354] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:49.160604 29331 rpcz_store.cc:276] 0812 19:07:47.369669 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.369807 (+ 138us) service_pool.cc:227] Handling call
0812 19:07:47.373387 (+ 3580us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.373605 (+ 218us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.373626 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.377631 (+ 4005us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.435547 (+ 57916us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:47.435585 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.435657 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.150778 (+1715121us) raft_consensus.cc:1671] finished
0812 19:07:49.150800 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.150962 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":57545,"threads_started":1,"wal-append.queue_time_us":103174}
I20260812 19:07:49.162758 29459 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:49.135185 1206 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.124142 1481 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.159109 29457 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:49.157464 29456 rpcz_store.cc:276] 0812 19:07:47.487499 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.060819 (+1573320us) service_pool.cc:227] Handling call
0812 19:07:49.145624 (+ 84805us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.109736 1193 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.168466 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:49.190413 29451 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:49.152856 1478 raft_consensus.cc:3032] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:49.150848 1451 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:49.181071 1477 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:49.181169 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 716346us
I20260812 19:07:49.153843 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 646) took 1383 ms. Trace:
I20260812 19:07:49.186725 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:49.181432 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 198499us
I20260812 19:07:49.222422 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:49.223304 29446 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:49.157214 1259 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:49.163496 30679 rpcz_store.cc:269] 0812 19:07:47.408635 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.413091 (+ 4456us) service_pool.cc:227] Handling call
0812 19:07:49.157774 (+1744683us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.232276 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2371) took 1748 ms. Trace:
I20260812 19:07:49.232687 29445 rpcz_store.cc:276] 0812 19:07:47.484315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.161261 (+1676946us) service_pool.cc:227] Handling call
0812 19:07:49.232217 (+ 70956us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.156504 1295 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.225109 1451 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:49.151146 1449 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:49.235580 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 130419us
I20260812 19:07:49.236387 30629 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:49.193882 1478 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:49.145597 1470 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.153277 1330 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.241998 29445 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:49.240870 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2386) took 1752 ms. Trace:
I20260812 19:07:49.182796 1081 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:49.183145 1481 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.242568 29456 rpcz_store.cc:276] 0812 19:07:47.488382 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.176651 (+1688269us) service_pool.cc:227] Handling call
0812 19:07:49.236358 (+ 59707us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:49.236540 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:49.236564 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:49.236603 (+ 39us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:49.240636 (+ 4033us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.240703 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.240812 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":124160}
I20260812 19:07:49.162415 1025 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.182210 1495 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:49.246770 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2387) took 1758 ms. Trace:
I20260812 19:07:49.250236 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2388) took 1761 ms. Trace:
I20260812 19:07:49.250882 29446 rpcz_store.cc:276] 0812 19:07:47.488664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.230520 (+1741856us) service_pool.cc:227] Handling call
0812 19:07:49.239501 (+ 8981us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:49.239663 (+ 162us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:49.239685 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:49.239721 (+ 36us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:49.246521 (+ 6800us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.246592 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.246709 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.251610 29445 rpcz_store.cc:276] 0812 19:07:47.489020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.234326 (+1745306us) service_pool.cc:227] Handling call
0812 19:07:49.241785 (+ 7459us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:49.249936 (+ 8151us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.250174 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.146854 1434 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.228943 1489 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.064736 30670 rpcz_store.cc:276] 0812 19:07:47.317584 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.318728 (+ 1144us) service_pool.cc:227] Handling call
0812 19:07:47.367686 (+ 48958us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.385871 (+ 18185us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.385907 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.386435 (+ 528us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.586757 (+200322us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:47.596030 (+ 9273us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.605060 (+ 9030us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:48.521204 (+916144us) raft_consensus.cc:1671] finished
0812 19:07:48.521230 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:48.535052 (+ 13822us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":191229,"threads_started":1,"wal-append.queue_time_us":1116240}
I20260812 19:07:49.265813 29451 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:49.266448 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 711) took 1783 ms. Trace:
I20260812 19:07:49.266736 29451 rpcz_store.cc:276] 0812 19:07:47.483111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.168187 (+1685076us) service_pool.cc:227] Handling call
0812 19:07:49.266385 (+ 98198us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.219507 29574 rpcz_store.cc:276] 0812 19:07:47.765576 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.772402 (+ 6826us) service_pool.cc:227] Handling call
0812 19:07:49.146894 (+1374492us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.141155 30674 rpcz_store.cc:276] 0812 19:07:47.147214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.171926 (+ 24712us) service_pool.cc:227] Handling call
0812 19:07:47.172313 (+ 387us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.172751 (+ 438us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.172783 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.218323 (+ 45540us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b2c000f7778. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:47.218765 (+ 442us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.353174 (+134409us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:47.356530 (+ 3356us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.367892 (+ 11362us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.099404 (+1731512us) raft_consensus.cc:1671] finished
0812 19:07:49.099445 (+ 41us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.113340 (+ 13895us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":230,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":118,"spinlock_wait_cycles":2813312,"thread_start_us":123248,"threads_started":1,"wal-append.queue_time_us":295885}
I20260812 19:07:49.278645 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:49.276573 1485 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.279663 29331 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 4
I20260812 19:07:49.280617 29456 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:49.285594 29567 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260812 19:07:49.287967 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 289410us
I20260812 19:07:49.283048 1490 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:49.289721 30625 connection.cc:232] RPC call timeout handler was delayed by -1.1761585108470172s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
I20260812 19:07:49.278059 1141 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:49.228533 1477 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:49.349272 29446 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260812 19:07:49.277590 1025 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:49.268738 1495 raft_consensus.cc:686] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:49.352936 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2395) took 1685 ms. Trace:
I20260812 19:07:49.360805 29446 rpcz_store.cc:276] 0812 19:07:47.667532 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.252295 (+1584763us) service_pool.cc:227] Handling call
0812 19:07:49.262370 (+ 10075us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:49.352639 (+ 90269us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.352873 (+ 234us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:49.348320 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1954) took 2131 ms. Trace:
W20260812 19:07:49.367766 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 369841us
W20260812 19:07:49.365816 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 392017us
I20260812 19:07:49.343459 1440 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.342628 1293 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:49.351457 709 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:49.350816 1259 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:49.362183 29453 rpcz_store.cc:276] 0812 19:07:47.209602 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.209778 (+ 176us) service_pool.cc:227] Handling call
0812 19:07:47.254407 (+ 44629us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:47.306856 (+ 52449us) raft_consensus.cc:1520] Early marking committed up to index 3
0812 19:07:47.578774 (+271918us) log.cc:842] Serialized 14 byte log entry
0812 19:07:49.340897 (+1762123us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:49.340946 (+ 49us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:49.340996 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.341066 (+ 70us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.341190 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17017,"prepare.queue_time_us":49153,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":134871,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":28322,"spinlock_wait_cycles":19751168,"thread_start_us":1897357,"threads_started":3,"wal-append.queue_time_us":1403974}
I20260812 19:07:49.352248 1141 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:49.378628 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 705) took 1817 ms. Trace:
I20260812 19:07:49.364821 708 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.369041 1083 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.269229 1489 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:49.458420 29399 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.26.205.126:35929, user_credentials={real_user=slave}} blocked reactor thread for 63671.2us
I20260812 19:07:49.374969 1505 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.459729 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:50.029913 29457 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s user 0.003s sys 0.007s
I20260812 19:07:49.363535 1497 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:49.471895 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:49.917140 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.004s sys 0.010s
I20260812 19:07:49.476922 30686 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
W20260812 19:07:49.993199 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.007s sys 0.009s
W20260812 19:07:50.044330 29457 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s user 0.003s sys 0.007s
W20260812 19:07:50.047058 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.667s user 0.005s sys 0.011s
I20260812 19:07:49.375281 1478 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.352919 1490 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.394913 1448 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:50.054859 1448 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.056567 1083 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.057147 1448 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.057540 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2369) took 2573 ms. Trace:
I20260812 19:07:49.473251 1459 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:49.403716 763 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:49.462275 709 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:49.396175 1259 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:49.476511 1404 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 1.804s user 0.006s sys 0.014s
W20260812 19:07:49.870252 30625 leader_election.cc:334] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.051s (SENT)
I20260812 19:07:49.477253 1479 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:49.477183 1442 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:49.476254 1470 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:49.477648 1472 log.cc:924] Time spent T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.218s user 0.000s sys 0.001s
I20260812 19:07:49.477694 1503 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:50.057210 1083 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:49.722682 30679 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260812 19:07:49.477759 1520 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:49.476300 1502 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:49.963832 1477 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:49.897168 1330 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.842834 1295 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.068199 1459 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.124040 1429 raft_consensus.cc:3032] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:50.124455 1429 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:49.911758 1449 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.125437 1526 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:49.871477 1140 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.126001 1526 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.126142 1140 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.127157 1489 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:50.026523 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.634s user 0.042s sys 0.057s
W20260812 19:07:50.029498 29282 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.551s user 0.001s sys 0.007s
W20260812 19:07:50.128137 29282 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.650s user 0.002s sys 0.007s
I20260812 19:07:50.125228 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:50.011597 1544 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.130627 29339 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:49.392870 29450 rpcz_store.cc:276] 0812 19:07:47.418435 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.427941 (+ 9506us) service_pool.cc:227] Handling call
0812 19:07:47.462006 (+ 34065us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.674074 (+212068us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.674104 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.697043 (+ 22939us) log.cc:842] Serialized 26 byte log entry
0812 19:07:48.259325 (+562282us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:48.435087 (+175762us) log.cc:842] Serialized 14 byte log entry
0812 19:07:49.235541 (+800454us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:49.235626 (+ 85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.235669 (+ 43us) raft_consensus.cc:1671] finished
0812 19:07:49.235681 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:49.235796 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":63527,"prepare.queue_time_us":175561,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":2069444,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":73602,"thread_start_us":937197,"threads_started":3,"wal-append.queue_time_us":1369758}
I20260812 19:07:50.129014 29452 raft_consensus.cc:2362] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3. Re-sending same reply.
I20260812 19:07:50.131788 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2404) took 1467 ms. Trace:
I20260812 19:07:49.468479 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:50.032106 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 2.1384e+06us
I20260812 19:07:49.287230 1485 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:50.032586 29283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.535s user 0.004s sys 0.004s
I20260812 19:07:50.028033 1514 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.120803 1502 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.136138 1140 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.023475 1434 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.136466 1502 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.136312 1514 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.136909 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.137200 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.137838 29342 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:50.138612 1514 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.135821 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:50.139441 29326 raft_consensus.cc:2362] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6. Re-sending same reply.
I20260812 19:07:50.139926 1485 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:49.551185 1487 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.292s user 0.000s sys 0.000s
I20260812 19:07:50.008940 1529 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.142015 1529 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.143177 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.144253 1529 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.044232 29453 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:49.693562 1443 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.525484 1206 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:50.145453 1414 log.cc:924] Time spent T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 2.380s user 0.007s sys 0.028s
I20260812 19:07:50.058002 29457 rpcz_store.cc:276] 0812 19:07:47.483730 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.176298 (+1692568us) service_pool.cc:227] Handling call
0812 19:07:49.230517 (+ 54219us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:49.258770 (+ 28253us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:49.258803 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:49.262273 (+ 3470us) log.cc:842] Serialized 26 byte log entry
0812 19:07:50.047570 (+785297us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:50.047610 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.047679 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.057330 (+ 9651us) raft_consensus.cc:1671] finished
0812 19:07:50.057351 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.057481 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":24546,"raft.queue_time_us":25628,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":47140,"thread_start_us":764022,"threads_started":1,"wal-append.queue_time_us":744590}
I20260812 19:07:50.067565 1259 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:49.405109 1293 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:49.546890 1507 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:50.099499 29331 consensus_meta.cc:287] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.819s user 0.007s sys 0.003s
I20260812 19:07:50.156898 1434 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:50.136569 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.043660 1519 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:50.093513 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:49.888711 30674 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:07:50.064625 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.074316 1453 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.062270 1539 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.132084 29452 rpcz_store.cc:276] 0812 19:07:48.664257 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.171532 (+507275us) service_pool.cc:227] Handling call
0812 19:07:50.125083 (+953551us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:50.131734 (+ 6651us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8380160}
I20260812 19:07:50.066045 1506 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:07:50.133241 29283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.655s user 0.005s sys 0.005s
W20260812 19:07:50.047683 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.653s user 0.007s sys 0.009s
I20260812 19:07:50.142412 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2106) took 2826 ms. Trace:
I20260812 19:07:50.070295 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:50.132447 29332 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:50.135876 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:50.127857 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.735s user 0.043s sys 0.057s
I20260812 19:07:50.145486 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.102114 1451 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:49.855043 1495 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.155761 29457 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:07:50.099900 1479 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.125672 1459 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.073545 709 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:50.233573 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 165452us
I20260812 19:07:50.302662 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.304050 1453 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:50.157191 29331 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:50.306138 29453 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260812 19:07:50.156759 1507 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:50.306793 1530 log.cc:924] Time spent T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.158s user 0.000s sys 0.000s
I20260812 19:07:50.306272 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 732) took 1028 ms. Trace:
I20260812 19:07:50.342092 29331 rpcz_store.cc:276] 0812 19:07:49.278224 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.278370 (+ 146us) service_pool.cc:227] Handling call
0812 19:07:50.306202 (+1027832us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.341506 1539 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.308753 1083 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:50.309335 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 262726us
I20260812 19:07:50.344043 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:50.345206 1477 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:50.345275 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.345775 1477 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:07:50.305996 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.746s user 0.000s sys 0.010s
W20260812 19:07:50.346366 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.059s user 0.000s sys 0.011s
I20260812 19:07:50.059796 1505 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.346801 1477 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.348109 29451 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:07:50.343990 1361 raft_consensus.cc:3032] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:50.062670 1497 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.342267 1502 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:50.309788 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 917233us
I20260812 19:07:50.233853 1438 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.351186 1502 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.311616 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 647) took 2543 ms. Trace:
I20260812 19:07:50.310901 1526 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.351750 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:50.308296 29569 rpcz_store.cc:276] 0812 19:07:47.316234 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.350423 (+ 34189us) service_pool.cc:227] Handling call
0812 19:07:47.361852 (+ 11429us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.362359 (+ 507us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.362389 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.362826 (+ 437us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.561005 (+198179us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:47.775968 (+214963us) log.cc:842] Serialized 14 byte log entry
0812 19:07:47.776242 (+ 274us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.776320 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:49.276282 (+1499962us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:50.142018 (+865736us) raft_consensus.cc:1671] finished
0812 19:07:50.142117 (+ 99us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.142236 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":74000,"prepare.queue_time_us":461257,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1531,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":22833,"spinlock_wait_cycles":35261568,"thread_start_us":404373,"threads_started":2,"wal-append.queue_time_us":238150}
I20260812 19:07:50.122623 1442 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.327653 1330 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:50.315057 1140 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.310509 1490 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.338219 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2105) took 3022 ms. Trace:
I20260812 19:07:50.310835 1193 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.307978 709 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.309685 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 707) took 2830 ms. Trace:
I20260812 19:07:50.350871 1438 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.120651 1450 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.351692 29580 rpcz_store.cc:276] 0812 19:07:47.768588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.900744 (+132156us) service_pool.cc:227] Handling call
0812 19:07:50.310940 (+2410196us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:50.311128 (+ 188us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.311309 (+ 181us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:50.311333 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:50.311373 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:50.311407 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.311456 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.311546 (+ 90us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":613356288}
I20260812 19:07:50.355119 29451 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:50.387899 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1976) took 1014 ms. Trace:
I20260812 19:07:50.388274 29451 rpcz_store.cc:276] 0812 19:07:49.373480 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.349939 (+976459us) service_pool.cc:227] Handling call
0812 19:07:50.354801 (+ 4862us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.387667 (+ 32866us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.387845 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.389421 29451 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:50.352396 29567 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.348920 1361 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:50.396662 1361 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.397812 1361 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.402434 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1967) took 1031 ms. Trace:
I20260812 19:07:50.402844 29446 rpcz_store.cc:276] 0812 19:07:49.371128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.375709 (+ 4581us) service_pool.cc:227] Handling call
0812 19:07:49.393639 (+ 17930us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:49.475115 (+ 81476us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:49.475147 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:49.501557 (+ 26410us) log.cc:842] Serialized 26 byte log entry
0812 19:07:50.249896 (+748339us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:50.249936 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.250029 (+ 93us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.402230 (+152201us) raft_consensus.cc:1671] finished
0812 19:07:50.402246 (+ 16us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.402375 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":68736,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"thread_start_us":403222,"threads_started":1,"wal-append.queue_time_us":698976}
I20260812 19:07:50.404991 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1978) took 1031 ms. Trace:
I20260812 19:07:50.405308 29446 rpcz_store.cc:276] 0812 19:07:49.373939 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.404188 (+1030249us) service_pool.cc:227] Handling call
0812 19:07:50.404512 (+ 324us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.404697 (+ 185us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:50.404721 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:50.404763 (+ 42us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:50.404800 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.404847 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.404938 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.360195 1515 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260812 19:07:50.360790 1025 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.997s user 0.010s sys 0.000s
I20260812 19:07:50.363454 1141 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.129972 1503 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.408711 1141 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.410192 1503 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.248441 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.410950 29450 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:50.415053 1025 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:50.371995 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 213892us
I20260812 19:07:50.372376 29452 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:50.375550 29576 rpcz_store.cc:276] 0812 19:07:47.315930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.316350 (+ 420us) service_pool.cc:227] Handling call
0812 19:07:47.338233 (+ 21883us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:48.465035 (+1126802us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.337916 (+1872881us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b540009e554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:50.338153 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2038834944}
I20260812 19:07:50.389940 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1977) took 1016 ms. Trace:
I20260812 19:07:50.377633 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.416217 29451 rpcz_store.cc:276] 0812 19:07:49.373710 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.388985 (+1015275us) service_pool.cc:227] Handling call
0812 19:07:50.389267 (+ 282us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.389759 (+ 492us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.389896 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.418465 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:50.380129 29458 rpcz_store.cc:276] 0812 19:07:47.479233 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.484602 (+ 5369us) service_pool.cc:227] Handling call
0812 19:07:47.648835 (+164233us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:47.649594 (+ 759us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.309492 (+2659898us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400323f50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:50.309628 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":376768384}
I20260812 19:07:50.419693 29331 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.421468 29451 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260812 19:07:50.346101 1448 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260812 19:07:50.427906 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 60545.3us
I20260812 19:07:50.432837 29331 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:50.438485 29567 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:50.440639 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.375442 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1975) took 1002 ms. Trace:
W20260812 19:07:50.417533 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 72330.5us
I20260812 19:07:50.423228 29458 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:50.415597 1025 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.382879 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1971) took 1003 ms. Trace:
I20260812 19:07:50.443286 29457 rpcz_store.cc:276] 0812 19:07:49.372256 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.147057 (+774801us) service_pool.cc:227] Handling call
0812 19:07:50.147314 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.375464 (+228150us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.375673 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.444851 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.442428 29453 rpcz_store.cc:276] 0812 19:07:49.373239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.322870 (+949631us) service_pool.cc:227] Handling call
0812 19:07:50.374880 (+ 52010us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.375094 (+ 214us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:50.375122 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:50.375164 (+ 42us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:50.375201 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.375254 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.375378 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.441354 29335 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:07:50.345201 1539 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.442134 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.365175 1330 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:50.381657 29282 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:50.440760 30670 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:50.493036 30629 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:50.444972 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.445832 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.445390 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.510141 30629 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:50.511807 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 130257us
I20260812 19:07:50.510150 30676 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:50.514658 29569 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:50.517814 30629 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:50.483386 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 777) took 1134 ms. Trace:
I20260812 19:07:50.518611 1551 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:50.519467 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.516837 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:50.519553 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.341450 1506 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.520612 30685 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:50.519656 29578 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:50.494431 1330 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.482170 1502 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:50.125317 1429 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.383306 1434 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.516918 1513 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:50.523689 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:50.524118 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 653874us
I20260812 19:07:50.524673 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 838) took 3188 ms. Trace:
I20260812 19:07:50.506114 1490 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.509734 1477 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:07:50.515766 1141 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:07:50.544899 1141 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.545300 1141 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:50.545651 1442 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.546167 1438 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:50.547174 1473 log.cc:924] Time spent T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 1.447s user 0.017s sys 0.003s
I20260812 19:07:50.525846 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1980) took 1122 ms. Trace:
I20260812 19:07:50.549243 29450 rpcz_store.cc:276] 0812 19:07:49.403186 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.411534 (+1008348us) service_pool.cc:227] Handling call
0812 19:07:50.411866 (+ 332us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:50.412234 (+ 368us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:50.412259 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:50.412699 (+ 440us) log.cc:842] Serialized 26 byte log entry
0812 19:07:50.441883 (+ 29184us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:50.441921 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.442005 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.525660 (+ 83655us) raft_consensus.cc:1671] finished
0812 19:07:50.525678 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.525792 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1042,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":28776,"threads_started":1,"wal-append.queue_time_us":26539}
W20260812 19:07:50.528550 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 86502.1us
I20260812 19:07:50.386505 1140 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.490731 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.520207 30684 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:50.532809 29569 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.532640 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.523212 1551 raft_consensus.cc:686] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:50.531142 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.535521 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.552069 30685 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:50.345764 1519 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.533241 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.535358 30675 rpcz_store.cc:276] 0812 19:07:47.306579 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.308835 (+ 2256us) service_pool.cc:227] Handling call
0812 19:07:47.325490 (+ 16655us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.360317 (+ 34827us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.368118 (+ 7801us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.415441 (+ 47323us) log.cc:842] Serialized 26 byte log entry
0812 19:07:47.565068 (+149627us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:47.575480 (+ 10412us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:47.582336 (+ 6856us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.144841 (+2562505us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:50.156733 (+ 11892us) raft_consensus.cc:1671] finished
0812 19:07:50.156761 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.483318 (+326557us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17279,"raft.queue_time_us":33742,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"spinlock_wait_cycles":10213760,"thread_start_us":96456,"threads_started":1,"wal-append.queue_time_us":212093}
I20260812 19:07:50.553586 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.540167 1505 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.541359 1470 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:50.542155 1362 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260812 19:07:50.542541 1474 log.cc:924] Time spent T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 1.446s user 0.003s sys 0.017s
I20260812 19:07:50.542882 1206 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:50.543198 29459 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:50.543795 1541 consensus_queue.cc:1046] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260812 19:07:50.521881 1521 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:50.552353 30671 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:50.157279 1544 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:50.553794 30625 connection.cc:232] RPC call timeout handler was delayed by -1.0825927269179374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260812 19:07:50.559989 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 712) took 3076 ms. Trace:
I20260812 19:07:50.528213 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 778) took 1163 ms. Trace:
I20260812 19:07:50.557238 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:50.558344 29576 raft_consensus.cc:2354] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:50.536484 1513 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:50.536639 1176 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:50.572209 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.561808 29459 rpcz_store.cc:276] 0812 19:07:47.483421 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.172417 (+1688996us) service_pool.cc:227] Handling call
0812 19:07:50.542727 (+1370310us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:50.542903 (+ 176us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.559741 (+ 16838us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.559936 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8103680}
I20260812 19:07:50.538962 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.518963 30674 rpcz_store.cc:276] 0812 19:07:49.349257 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.361909 (+ 12652us) service_pool.cc:227] Handling call
0812 19:07:49.479093 (+117184us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.382573 (+903480us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.481504 (+ 98931us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:50.563962 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.565991 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.570216 30679 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:50.602335 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:50.603046 30679 raft_consensus.cc:2354] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:50.603832 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.571257 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:50.445057 1450 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.576535 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.605471 29578 raft_consensus.cc:2354] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:50.606207 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.607245 30671 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:50.607854 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.578454 1506 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.608455 29567 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:50.608505 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.579553 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.580207 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.571926 1521 raft_consensus.cc:686] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:50.582165 706 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:07:50.578922 1434 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.582835 1438 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:50.584525 1519 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:50.564054 30670 rpcz_store.cc:276] 0812 19:07:49.364834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.368816 (+ 3982us) service_pool.cc:227] Handling call
0812 19:07:50.344471 (+975655us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:50.344708 (+ 237us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.505896 (+161188us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.521744 (+ 15848us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":130501888}
I20260812 19:07:50.659611 1438 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:50.586020 1025 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.586203 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.660668 29576 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:50.660565 1438 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.661605 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.662750 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:50.663275 29576 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.586347 1141 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:50.664717 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.665840 1141 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.666074 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.587414 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.666936 30675 raft_consensus.cc:2396] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:07:50.587692 1140 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.590842 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2111) took 3116 ms. Trace:
I20260812 19:07:50.591099 1507 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.591388 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.592914 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:50.594250 29575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2109) took 3214 ms (client timeout 1653 ms). Trace:
I20260812 19:07:50.594733 1551 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.595391 1479 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.572410 1176 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:50.682617 1176 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.683880 1176 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:50.572649 1513 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.571599 1544 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.596618 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.690402 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.691470 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.692174 29326 raft_consensus.cc:2354] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:50.693251 29530 leader_election.cc:398] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:50.553272 1490 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.598003 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.600013 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:50.600715 1552 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:07:50.698658 29450 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:50.564165 29334 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:50.563716 1449 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260812 19:07:50.561086 1172 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260812 19:07:50.609280 29337 raft_consensus.cc:2379] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:50.614362 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.702811 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 650) took 2867 ms. Trace:
I20260812 19:07:50.705425 29335 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:50.620883 29569 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:07:50.607286 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:50.644855 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 77828us
W20260812 19:07:50.652431 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 177300us
I20260812 19:07:50.653846 30686 raft_consensus.cc:2379] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:50.653671 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.608506 30671 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:50.558521 29574 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:50.654146 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.708292 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.709365 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.709877 29458 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:50.654668 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.710709 29458 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "a34c47dd29c14748a018d716ac6c39df"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:50.711360 29458 raft_consensus.cc:595] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Received request to transfer leadership to TS a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.711627 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.712251 29567 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:50.713047 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.655346 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.655839 29281 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:50.714901 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 59164.7us
I20260812 19:07:50.670135 30623 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:50.716238 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:50.716547 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:07:50.716790 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46754.7us
I20260812 19:07:50.716928 30671 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:07:50.671468 30685 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:50.717794 1449 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:50.718272 1449 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.718642 1449 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:50.718739 1495 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:50.719128 1495 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.719466 1495 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.722402 1490 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:50.703722 1544 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:50.723964 1544 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:50.724507 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1969) took 1352 ms. Trace:
I20260812 19:07:50.717819 1526 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.725528 1526 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.726277 1450 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.727337 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1974) took 1354 ms. Trace:
I20260812 19:07:50.727519 1526 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:50.674804 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 102673us
I20260812 19:07:50.728518 29399 leader_election.cc:398] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:50.728788 29399 leader_election.cc:401] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260812 19:07:50.673806 29575 rpcz_store.cc:269] 0812 19:07:47.379800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.405827 (+ 26027us) service_pool.cc:227] Handling call
0812 19:07:50.594034 (+3188207us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400156654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:50.594157 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3574397824}
I20260812 19:07:50.693552 29530 leader_election.cc:401] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:50.716295 29281 leader_election.cc:398] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:50.655033 1505 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:50.691494 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:50.699298 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 99355.2us
I20260812 19:07:50.701297 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:50.655524 30679 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:50.671756 29572 rpcz_store.cc:276] 0812 19:07:47.474766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.754125 (+279359us) service_pool.cc:227] Handling call
0812 19:07:47.766392 (+ 12267us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:47.766763 (+ 371us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:47.766792 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:47.767245 (+ 453us) log.cc:842] Serialized 26 byte log entry
0812 19:07:48.972101 (+1204856us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:48.972142 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:48.972209 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.300453 (+1328244us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b48002301d8. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6ecb57f1 00007f9c6ecac8e7 00007f9c6ecac77a 00007f9c6eda095f 00007f9c6eda0627 00007f9c6edb4b2c 00007f9c6edb4ad8 00007f9c6edb4a58 00007f9c6edb4a13 00007f9c6edb3cbc 00007f9c6dd35be6
0812 19:07:50.541945 (+241492us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:50.590650 (+ 48705us) raft_consensus.cc:1671] finished
0812 19:07:50.590674 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.590787 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1079,"raft.run_cpu_time_us":36680,"raft.run_wall_time_us":2526809,"spinlock_wait_cycles":36747520,"thread_start_us":1204487,"threads_started":1,"wal-append.queue_time_us":1241604}
I20260812 19:07:50.673821 29580 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:50.729480 29399 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:50.729866 29452 rpcz_store.cc:276] 0812 19:07:49.372961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.314075 (+941114us) service_pool.cc:227] Handling call
0812 19:07:50.345513 (+ 31438us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.415916 (+ 70403us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.727140 (+311224us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b540006d6d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:50.727286 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":60929280}
I20260812 19:07:50.739198 29530 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.724543 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.724869 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:50.741915 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.724871 29445 rpcz_store.cc:276] 0812 19:07:49.371729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.371851 (+ 122us) service_pool.cc:227] Handling call
0812 19:07:49.372162 (+ 311us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.351731 (+979569us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:50.352012 (+ 281us) log.cc:842] Serialized 14 byte log entry
0812 19:07:50.434382 (+ 82370us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:50.434427 (+ 45us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:50.434480 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.724275 (+289795us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b54001ac554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:50.724318 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.724450 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1784,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":405,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":2252363392,"thread_start_us":81969,"threads_started":1,"wal-append.queue_time_us":82027}
I20260812 19:07:50.736738 29527 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.730585 29281 leader_election.cc:401] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:50.735966 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:50.746536 29450 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 9
I20260812 19:07:50.745800 29281 leader_election.cc:398] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:50.747332 29281 leader_election.cc:401] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:07:50.749073 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.751065 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.752905 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.756560 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.758199 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.760699 29527 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.732587 29403 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:50.718145 709 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.763275 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.705755 29577 rpcz_store.cc:276] 0812 19:07:47.834953 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:48.775903 (+940950us) service_pool.cc:227] Handling call
0812 19:07:48.957780 (+181877us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:49.241733 (+283953us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:49.241763 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:49.262521 (+ 20758us) log.cc:842] Serialized 26 byte log entry
0812 19:07:49.378374 (+115853us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:49.378407 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.694288 (+1315881us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400135454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:50.694341 (+ 53us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:50.694390 (+ 49us) raft_consensus.cc:1671] finished
0812 19:07:50.694405 (+ 15us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.702748 (+ 8343us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1039,"raft.queue_time_us":325977,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":208892,"spinlock_wait_cycles":490186624,"thread_start_us":102493,"threads_started":2,"wal-append.queue_time_us":767777}
I20260812 19:07:50.765164 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.766739 29282 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:50.764928 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:50.774684 29458 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:50.764391 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.767818 29527 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.760825 1515 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:50.777664 29527 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:50.779510 708 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:50.779733 1448 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:50.780193 708 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:50.780345 1448 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.771358 29399 leader_election.cc:398] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
I20260812 19:07:50.781764 708 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:50.782297 29399 leader_election.cc:401] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:50.782536 1448 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.783069 1503 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3.
I20260812 19:07:50.785305 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:50.790570 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 74112.3us
I20260812 19:07:50.743752 1477 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:50.708312 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.748287 1450 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:07:50.748473 1176 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4.
I20260812 19:07:50.761411 1507 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:50.761713 1552 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:50.758384 1445 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:50.793519 1448 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:50.770486 1479 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:50.799372 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.762019 1455 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:50.799647 1482 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:50.800640 29282 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:50.800818 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.802208 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.770761 1489 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:50.799558 1025 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:07:50.806891 29281 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:50.770942 1463 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:50.771202 1434 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:50.806344 1330 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.809363 1292 consensus_queue.cc:1035] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:50.810237 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.810935 1259 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:50.811851 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:50.812433 29335 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:50.716944 1513 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.771060 1505 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:50.819746 1541 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:50.724859 1544 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.821769 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:50.864048 30681 raft_consensus.cc:1226] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:50.864794 29446 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
W20260812 19:07:50.866315 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 44609.9us
I20260812 19:07:50.794922 1140 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:50.796479 1450 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.801635 1141 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:50.824586 1503 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:50.802740 1507 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:50.827014 1580 log.cc:924] Time spent T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
I20260812 19:07:50.796085 709 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.870376 29281 leader_election.cc:398] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:50.879176 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.827466 29335 raft_consensus.cc:595] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.870199 1061 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:50.886811 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:50.870491 1513 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:50.871243 30625 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:50.822718 1463 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:50.873469 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 72741.6us
I20260812 19:07:50.824292 1434 raft_consensus.cc:686] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:50.828675 1515 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6.
I20260812 19:07:50.762445 1521 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.829596 1259 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.872233 1141 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:50.887259 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 645) took 3141 ms. Trace:
I20260812 19:07:50.869426 1140 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:50.872081 1450 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:50.872038 1206 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:50.899279 29399 leader_election.cc:398] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7.
I20260812 19:07:50.899910 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.828418 1541 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:50.891551 763 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:50.878624 1507 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:07:50.901091 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 74418.8us
I20260812 19:07:50.902995 29579 rpcz_store.cc:276] 0812 19:07:47.745523 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.841935 (+ 96412us) service_pool.cc:227] Handling call
0812 19:07:50.886536 (+3044601us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:50.886755 (+ 219us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.886925 (+ 170us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:50.886949 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:50.886992 (+ 43us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:50.887025 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:50.887078 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:50.887192 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7803136}
I20260812 19:07:50.871968 1460 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260812 19:07:50.868821 1505 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:50.894953 1448 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:50.927891 29453 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:50.895403 1463 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.904505 1141 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.904625 1140 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:50.902943 1259 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:50.953146 29445 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
W20260812 19:07:50.943588 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 49756.2us
I20260812 19:07:50.958201 29445 raft_consensus.cc:595] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.958516 29445 raft_consensus.cc:603] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:07:50.955122 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:50.924299 1541 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.926028 1534 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:50.922058 1206 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:50.930760 1513 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:50.961052 29335 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260812 19:07:50.963871 29335 pending_rounds.cc:76] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Aborting all ops after (but not including) 6
I20260812 19:07:50.964133 29335 pending_rounds.cc:97] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260812 19:07:50.966018 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:50.892421 1362 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260812 19:07:50.964766 29335 raft_consensus.cc:2866] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:50.905362 1176 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260812 19:07:50.980459 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 109280us
I20260812 19:07:50.986577 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:50.988131 29399 leader_election.cc:398] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7.
I20260812 19:07:50.988394 29399 leader_election.cc:401] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260812 19:07:50.989568 1330 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:50.990640 1442 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7.
I20260812 19:07:50.964130 1539 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:50.968674 1206 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:50.938512 1448 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.008934 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:51.011711 30623 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:50.981573 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:50.936710 1434 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:51.020689 1596 log.cc:924] Time spent T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
W20260812 19:07:51.021054 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 62401.5us
I20260812 19:07:51.017100 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:51.025565 30625 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.027284 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.028157 30625 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.020301 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.033535 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.033776 29399 leader_election.cc:398] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:51.034825 29399 leader_election.cc:401] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:07:51.025849 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:51.016708 706 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:51.035477 1575 log.cc:924] Time spent T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.163s user 0.001s sys 0.000s
W20260812 19:07:51.038031 1577 log.cc:924] Time spent T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.181s user 0.002s sys 0.000s
I20260812 19:07:51.043012 30629 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.023541 29574 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:07:51.073490 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 53281.7us
I20260812 19:07:51.029878 1292 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.048432 1442 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6.
I20260812 19:07:51.035804 709 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:51.082196 1292 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.082957 1448 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.084609 29450 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:51.085246 1557 raft_consensus.cc:2741] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260812 19:07:51.077883 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49783.2us
W20260812 19:07:51.047495 1599 log.cc:924] Time spent T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.083s user 0.002s sys 0.000s
I20260812 19:07:51.007793 1539 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.009745 1330 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260812 19:07:51.038992 1612 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:51.080504 1590 log.cc:924] Time spent T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260812 19:07:51.094700 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:51.099705 29576 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:51.075682 1526 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:51.088215 1025 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:51.048578 1537 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260812 19:07:51.088363 1502 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:07:51.125258 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 100349us
I20260812 19:07:51.124488 1502 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:51.131249 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.137486 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2175) took 1008 ms. Trace:
I20260812 19:07:51.137924 29337 rpcz_store.cc:276] 0812 19:07:50.128749 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.128877 (+ 128us) service_pool.cc:227] Handling call
0812 19:07:51.137241 (+1008364us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b540016fb54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:51.137407 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":686887552}
I20260812 19:07:51.139963 1496 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3.
I20260812 19:07:51.078127 706 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:51.124785 1597 log.cc:924] Time spent T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.075s user 0.000s sys 0.001s
I20260812 19:07:51.152429 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.156739 29282 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:51.156710 1532 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:51.159613 1539 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:51.163851 30625 connection.cc:232] RPC call timeout handler was delayed by -2.006366022862494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260812 19:07:51.131045 1502 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:51.164347 30625 connection.cc:232] RPC call timeout handler was delayed by -1.7588660230394453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260812 19:07:51.164893 30625 connection.cc:232] RPC call timeout handler was delayed by -1.5455660228617489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2222s for a response
I20260812 19:07:51.131771 1505 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.139410 1449 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.192246 29578 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:51.195961 1620 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:07:51.175246 1448 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:07:51.192823 1294 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.179s user 0.015s sys 0.024s
W20260812 19:07:51.208870 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 83004us
I20260812 19:07:51.209780 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.192181 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.181252 1361 consensus_queue.cc:1046] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
W20260812 19:07:51.207911 1294 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.199s user 0.016s sys 0.024s
I20260812 19:07:51.221714 706 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.222708 1449 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.224159 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.224756 29578 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:51.214679 1445 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:51.228617 29399 leader_election.cc:398] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9.
I20260812 19:07:51.216674 1330 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:51.211333 709 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260812 19:07:51.252555 1625 raft_consensus.cc:3032] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:51.253070 1625 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.261873 1620 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.262066 29578 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:51.253221 1293 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:51.265027 29399 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.267249 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.273563 30686 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 3 LEARNER]: Advancing to term 5
I20260812 19:07:51.267324 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.274881 30671 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:51.256145 1513 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:51.263741 1495 consensus_meta.cc:287] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.544s user 0.009s sys 0.002s
I20260812 19:07:51.276739 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:51.308027 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 46642.6us
I20260812 19:07:51.276381 1293 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260812 19:07:51.325084 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 48452.7us
I20260812 19:07:51.325986 29399 leader_election.cc:398] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9.
I20260812 19:07:51.326382 29399 leader_election.cc:401] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260812 19:07:51.328022 30625 leader_election.cc:334] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.710s (SENT)
I20260812 19:07:51.328531 30625 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.335494 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2393) took 3757 ms. Trace:
I20260812 19:07:51.335876 29456 rpcz_store.cc:276] 0812 19:07:47.581590 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:49.243737 (+1662147us) service_pool.cc:227] Handling call
0812 19:07:49.261164 (+ 17427us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:49.281148 (+ 19984us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:51.335269 (+2054121us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b54002ff3d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:51.335437 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":41344896}
I20260812 19:07:51.337705 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 741) took 1108 ms. Trace:
I20260812 19:07:51.340042 29341 rpcz_store.cc:276] 0812 19:07:50.229729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.261779 (+ 32050us) service_pool.cc:227] Handling call
0812 19:07:51.337194 (+1075415us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b5800003398. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:51.337366 (+ 172us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:51.337498 (+ 132us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:51.337513 (+ 15us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:51.337540 (+ 27us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:51.337564 (+ 24us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:51.337598 (+ 34us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:51.337663 (+ 65us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3648384}
I20260812 19:07:51.330973 1434 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9.
I20260812 19:07:51.330119 1603 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:51.311080 1495 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.341143 1513 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:51.355388 30625 leader_election.cc:334] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.985s (SENT)
I20260812 19:07:51.356562 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.324870 1541 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.361928 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.353108 1603 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:51.365608 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 45931.8us
I20260812 19:07:51.369277 29337 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:51.371683 29342 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:51.373179 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.373817 29452 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:51.377877 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.367563 1603 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:07:51.405642 30625 leader_election.cc:334] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.222s (SENT)
I20260812 19:07:51.362308 1622 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:51.423873 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 51243.9us
I20260812 19:07:51.371340 1025 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:51.317072 1620 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.342880 1625 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:51.425861 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47689.3us
W20260812 19:07:51.430615 1629 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.086s user 0.000s sys 0.001s
W20260812 19:07:51.430760 1631 log.cc:924] Time spent T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.088s user 0.000s sys 0.001s
I20260812 19:07:51.438750 1330 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
W20260812 19:07:51.442343 1636 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.129s user 0.001s sys 0.000s
I20260812 19:07:51.442639 1259 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:51.444866 1450 consensus_meta.cc:287] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.518s user 0.002s sys 0.007s
I20260812 19:07:51.445569 1625 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.445315 1450 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.445717 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.454727 29450 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:51.447512 1141 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.447712 1603 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.448781 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.422513 1434 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.437292 1463 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.444799 1259 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:51.462474 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.470909 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.472004 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.474645 29450 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:51.476135 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.469197 29452 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:51.478837 1141 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.480733 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:51.482208 1643 log.cc:924] Time spent T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
W20260812 19:07:51.484009 1641 log.cc:924] Time spent T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260812 19:07:51.484447 30686 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 LEARNER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:51.485185 1450 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.485270 29335 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:51.485702 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.486186 1463 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.483786 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.487692 1603 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:51.488847 1648 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.134s user 0.001s sys 0.000s
I20260812 19:07:51.486881 29341 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260812 19:07:51.490326 1541 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:51.492566 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.489966 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.494606 29452 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:51.495544 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.502285 29459 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:51.504207 29282 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:51.507784 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.509250 29281 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:51.510298 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.510342 29445 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:51.513640 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.514364 29452 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 1.
I20260812 19:07:51.515268 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.518553 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.517273 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.519134 29580 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.520395 29452 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:51.521260 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.429539 1025 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.522044 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.517616 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.530447 30676 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:51.530759 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.531314 30671 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:07:51.529438 30625 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 3.724s (SENT) [suppressed 22 similar messages]
I20260812 19:07:51.527566 29457 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:51.534845 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.535506 29572 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:51.476814 29450 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:51.512887 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.537375 29459 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:51.531329 29451 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:51.475318 1330 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:51.512584 1361 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.547019 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.547468 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.526857 1541 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:51.558001 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.549647 1433 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:07:51.562155 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.562788 29459 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.566148 30671 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:51.571704 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.572052 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.576227 29459 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:51.578356 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 48959.7us
I20260812 19:07:51.574990 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.581171 29446 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:07:51.582031 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1411080588586628s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260812 19:07:51.582437 29399 connection.cc:232] RPC call timeout handler was delayed by -1.0790080588776618s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
I20260812 19:07:51.583669 29459 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:51.585418 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.586112 29459 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:51.587330 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.588558 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.588864 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.548936 1495 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:51.590898 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.608943 1433 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.609980 1433 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.610471 1632 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:51.611095 1632 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.613104 1632 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:51.576160 1507 consensus_meta.cc:287] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.635s user 0.011s sys 0.000s
I20260812 19:07:51.614566 1507 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.615887 29567 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:51.616961 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.617709 29445 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:51.618813 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.620970 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:51.622213 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.620960 30676 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:51.625017 29580 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:51.622781 29569 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:51.632802 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.629390 29282 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:51.617756 1507 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.627014 1434 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:51.589392 1541 raft_consensus.cc:686] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.636883 29446 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:51.650462 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.654564 29282 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:07:51.647230 1475 log.cc:924] Time spent T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 2.504s user 0.033s sys 0.051s
I20260812 19:07:51.651485 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.660409 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.657615 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.663051 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.654340 1470 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:51.664057 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.665251 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.666080 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:51.668663 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.664072 1470 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:51.657555 1639 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:07:51.655447 1434 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:51.675311 29399 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.675625 29569 raft_consensus.cc:2396] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260812 19:07:51.630089 1330 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260812 19:07:51.627781 1206 consensus_meta.cc:287] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.610s user 0.007s sys 0.004s
I20260812 19:07:51.642685 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:51.668687 1293 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:51.673794 1448 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.693102 1206 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.695217 1434 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:51.695338 29326 raft_consensus.cc:1226] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:51.697757 1206 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.699000 1505 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:51.699883 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.700338 1639 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.701539 1448 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:51.701366 1639 consensus_queue.cc:237] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.703022 1448 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:51.703970 1448 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:51.704339 1448 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:51.705106 1293 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:51.705617 1293 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:51.705183 1448 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.707474 1448 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:51.680488 1541 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.683948 29341 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260812 19:07:51.717801 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.718075 1544 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:51.720120 1497 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:51.721210 29528 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.468s user 0.009s sys 0.006s
W20260812 19:07:51.725270 1140 consensus_meta.cc:287] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.774s user 0.012s sys 0.000s
I20260812 19:07:51.725742 1140 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.728609 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:51.728881 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.729219 29446 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:51.732707 1497 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:07:51.734220 1606 log.cc:924] Time spent T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.657s user 0.000s sys 0.003s
I20260812 19:07:51.736792 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 1990) took 1173 ms. Trace:
I20260812 19:07:51.685722 1470 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.737182 29453 rpcz_store.cc:276] 0812 19:07:50.563574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.564760 (+ 1186us) service_pool.cc:227] Handling call
0812 19:07:50.625413 (+ 60653us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.936522 (+311109us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:51.736589 (+800067us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b5400310954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:51.736738 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":690267008}
I20260812 19:07:51.743299 29446 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:51.744709 29282 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:51.745677 1617 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:07:51.745853 1502 consensus_meta.cc:287] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.565s user 0.008s sys 0.002s
W20260812 19:07:51.760439 1602 log.cc:924] Time spent T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.685s user 0.000s sys 0.002s
I20260812 19:07:51.773550 1502 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:51.759119 1662 log.cc:924] Time spent T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.112s user 0.002s sys 0.000s
W20260812 19:07:51.747220 1607 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.663s user 0.003s sys 0.000s
I20260812 19:07:51.747931 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:51.850440 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2285) took 1716 ms. Trace:
I20260812 19:07:51.850812 29339 rpcz_store.cc:276] 0812 19:07:50.134054 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.134215 (+ 161us) service_pool.cc:227] Handling call
0812 19:07:51.747751 (+1613536us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b5800003398. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:51.850357 (+102606us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17088896}
I20260812 19:07:51.727794 1140 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:51.806073 29528 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.553s user 0.010s sys 0.011s
W20260812 19:07:51.856784 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 1.62427e+06us
I20260812 19:07:51.863583 29572 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
W20260812 19:07:51.865568 1651 log.cc:924] Time spent T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.219s user 0.000s sys 0.002s
W20260812 19:07:51.866484 29334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 740) took 1706 ms (client timeout 1638 ms). Trace:
W20260812 19:07:51.866847 29334 rpcz_store.cc:269] 0812 19:07:50.159781 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.159927 (+ 146us) service_pool.cc:227] Handling call
0812 19:07:51.866307 (+1706380us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003d7f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:51.866415 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":339455744}
I20260812 19:07:51.867811 29577 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:51.868201 30629 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.869349 1490 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:51.869710 1490 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:51.870115 1490 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:51.869855 1557 consensus_queue.cc:1046] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:07:51.870992 1557 raft_consensus.cc:3032] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:51.871306 1557 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:51.872053 1557 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.873148 1557 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:51.876215 30684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 787) took 1344 ms (client timeout 1590 ms). Trace:
I20260812 19:07:51.879899 29577 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260812 19:07:51.876864 30684 rpcz_store.cc:269] 0812 19:07:50.531752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.555312 (+ 23560us) service_pool.cc:227] Handling call
0812 19:07:51.875885 (+1320573us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540070e9d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:51.876130 (+ 245us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1499020672}
I20260812 19:07:51.897377 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:51.901405 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:51.765599 1497 consensus_queue.cc:1046] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260812 19:07:51.813274 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 848) took 1320 ms. Trace:
I20260812 19:07:51.809423 1617 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:51.820231 1622 consensus_queue.cc:1046] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260812 19:07:51.823002 1526 consensus_queue.cc:1046] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:51.824044 30685 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260812 19:07:51.867014 1293 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:51.831408 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.831923 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.832161 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.831503 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.833207 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.834785 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.836517 1025 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.837914 1478 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.839900 29326 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260812 19:07:51.841711 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.842757 29335 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:51.843818 1495 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:51.845005 1081 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:51.845836 30625 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:51.943989 29452 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:51.847338 29342 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
W20260812 19:07:51.847760 1667 log.cc:924] Time spent T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.083s user 0.004s sys 0.000s
W20260812 19:07:51.713886 29574 consensus_meta.cc:287] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.601s user 0.008s sys 0.000s
I20260812 19:07:51.904541 30681 rpcz_store.cc:276] 0812 19:07:50.493323 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.510412 (+ 17089us) service_pool.cc:227] Handling call
0812 19:07:50.521532 (+ 11120us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:50.869135 (+347603us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:51.813034 (+943899us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400677d54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:51.813214 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":787233536}
I20260812 19:07:51.913775 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:51.918116 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.826918 1429 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:51.961792 29528 connection.cc:232] RPC call timeout handler was delayed by -2.0625024940818548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260812 19:07:51.944725 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 98967.7us
W20260812 19:07:51.922653 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 410802us
I20260812 19:07:51.926637 29567 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:51.894737 1434 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.942061 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.963378 30625 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.971273 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.971954 30685 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:51.973156 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.902133 1470 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:51.975389 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:51.967133 29528 connection.cc:232] RPC call timeout handler was delayed by -1.73801614693366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260812 19:07:51.976503 29528 connection.cc:232] RPC call timeout handler was delayed by -1.6684256738517433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260812 19:07:51.976806 30625 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:51.976815 29528 connection.cc:232] RPC call timeout handler was delayed by -1.6162256740499288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
W20260812 19:07:51.977787 29528 connection.cc:232] RPC call timeout handler was delayed by -1.5946256739553064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260812 19:07:51.978072 29528 connection.cc:232] RPC call timeout handler was delayed by -1.339191413950175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260812 19:07:51.978317 29528 connection.cc:232] RPC call timeout handler was delayed by -1.2725941690150648s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2126s for a response
W20260812 19:07:51.978627 29528 connection.cc:232] RPC call timeout handler was delayed by -1.1686264069285244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260812 19:07:51.978577 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.979302 30676 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:51.980726 30681 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:51.983786 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:51.988054 709 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 19:07:51.988451 709 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:51.988793 709 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 1 FOLLOWER]: Advancing to term 2
I20260812 19:07:51.989382 29567 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:51.990084 29174 catalog_manager.cc:5495] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 4, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 4 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:51.994093 1622 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:51.996035 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.996172 1434 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:51.998292 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:51.997470 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.998050 1622 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:51.999316 1622 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:51.999809 1622 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.002310 1495 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:07:52.004029 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.006536 30625 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:51.912568 1617 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.010953 1628 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:52.011394 1628 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:52.012194 1628 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.006645 29572 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:51.969903 1537 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260812 19:07:52.012867 1490 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.013346 1628 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:51.929222 1502 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:51.998914 29575 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.014196 1502 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:52.014390 1618 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:51.974483 1455 consensus_queue.cc:1046] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260812 19:07:52.014679 1502 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:52.016026 1502 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.016135 1455 raft_consensus.cc:3032] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:52.016508 1455 raft_consensus.cc:729] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:51.974103 1526 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:52.017987 29337 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:51.965617 29574 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:52.020852 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2136) took 1075 ms. Trace:
I20260812 19:07:52.021203 29574 rpcz_store.cc:276] 0812 19:07:50.945000 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.948663 (+ 3663us) service_pool.cc:227] Handling call
0812 19:07:52.020791 (+1072128us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:52.023458 1495 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:52.023434 1458 consensus_queue.cc:1046] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:07:52.024262 1458 consensus_queue.cc:1225] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 19:07:52.024694 1544 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:52.025048 1544 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:52.025092 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.025861 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:52.010990 1622 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.027765 1025 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:52.025938 1544 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.028256 1025 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.028692 1544 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:52.026697 1502 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.030185 29446 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:52.030951 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.032063 30625 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:51.999508 1670 log.cc:924] Time spent T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.105s user 0.001s sys 0.000s
I20260812 19:07:52.032574 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.004268 1557 raft_consensus.cc:3032] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:52.034868 29567 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:51.950492 29453 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260812 19:07:51.985860 30671 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:52.039819 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:52.034391 29174 catalog_manager.cc:5495] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.043102 30676 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:52.044200 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.029081 1025 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.047619 1532 consensus_queue.cc:1046] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:51.925499 1160 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260812 19:07:52.048225 29174 catalog_manager.cc:5495] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.052122 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:52.017247 1455 consensus_queue.cc:260] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.055364 1507 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:07:52.048457 1541 consensus_queue.cc:1046] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260812 19:07:52.056015 998 consensus_queue.cc:1159] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6dd313c6592d4f33bab314a73f28b007" responder_term: 3 status { last_received { term: 3 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 3 index: 6 } }
I20260812 19:07:52.056672 1541 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:52.056999 1541 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.057762 1541 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.058525 1515 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:52.058732 1622 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.041263 1557 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:07:52.055296 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 59361.3us
I20260812 19:07:52.060016 1557 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.060582 29528 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.061218 1557 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:51.966421 1551 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260812 19:07:52.062127 1638 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:52.062733 1638 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.052018 29174 catalog_manager.cc:5495] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 4 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.061890 29528 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.066424 29334 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:52.067972 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.059031 1541 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:52.068562 29446 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.034279 1618 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:52.067503 29174 catalog_manager.cc:5495] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:07:52.073812 29281 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:52.035706 1526 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:52.073807 30629 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:52.041674 1539 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:52.043515 1537 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:52.074976 29174 catalog_manager.cc:5495] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 4 to 5, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 5 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.055794 1455 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.091998 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.088186 29575 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:52.087961 29174 catalog_manager.cc:5495] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:07:52.093367 29174 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.094744 29174 catalog_manager.cc:5495] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.096288 29174 catalog_manager.cc:5495] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.061931 1470 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:52.099152 29174 catalog_manager.cc:5495] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 4 to 7, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 7 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.068645 998 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:52.048815 1513 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260812 19:07:52.106276 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 74270.9us
I20260812 19:07:52.111143 29174 catalog_manager.cc:5495] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.087388 1507 raft_consensus.cc:2741] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260812 19:07:52.114604 29174 catalog_manager.cc:5495] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.063180 1450 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260812 19:07:52.084340 1526 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:52.090319 1537 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:52.059130 1515 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.063355 1140 consensus_queue.cc:1046] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260812 19:07:52.119236 29174 catalog_manager.cc:5495] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:07:52.137539 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:52.101416 1502 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.148913 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:52.151130 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 59206.5us
I20260812 19:07:52.143168 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.143486 29452 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:52.112136 1495 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:52.160523 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:52.161065 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 64426.4us
I20260812 19:07:52.163367 1618 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:52.163782 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47184.5us
I20260812 19:07:52.178642 29528 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.139098 1537 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.107439 1626 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:07:52.169394 1140 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:52.150509 29174 catalog_manager.cc:5495] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 5, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 5 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.185748 29174 catalog_manager.cc:5495] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 3 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } }
I20260812 19:07:52.187170 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.163235 1270 raft_consensus.cc:979] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: : Instructing follower dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:52.107476 1532 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:07:52.187804 29451 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.178573 29456 raft_consensus.cc:2362] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5. Re-sending same reply.
I20260812 19:07:52.188828 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:52.159888 30676 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:52.187637 29174 catalog_manager.cc:5495] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:52.110657 1628 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:07:52.194219 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.196035 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:52.194792 29458 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.176608 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.183341 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:52.172765 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 121110us
I20260812 19:07:52.108743 1525 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:52.174809 29446 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:52.191310 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:52.198653 30671 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.201009 29452 raft_consensus.cc:2379] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:52.192456 29174 catalog_manager.cc:5495] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 2 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:07:52.203457 29399 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.206796 30671 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:52.178316 1673 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:52.188452 998 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:07:52.219442 1172 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:52.211016 29282 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:52.211314 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.212417 29281 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:52.214494 29451 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:52.184998 1140 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.211038 1534 raft_consensus.cc:1067] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Signalling peer dd7e8c7a8bc54e73a39411cfc241fe9c to start an election
I20260812 19:07:52.230589 29458 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:52.192016 1293 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:52.234138 1679 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.138s user 0.001s sys 0.000s
I20260812 19:07:52.237609 29338 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:52.220003 1172 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.209079 1532 raft_consensus.cc:686] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:52.239722 1172 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:52.239459 1532 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.240589 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 799) took 1022 ms. Trace:
I20260812 19:07:52.209461 1330 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:52.202867 1525 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:52.238474 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49711.2us
I20260812 19:07:52.201470 1490 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.242241 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.245751 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.208920 1497 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:52.207345 1495 raft_consensus.cc:686] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:52.214848 1618 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.221184 1626 raft_consensus.cc:686] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.256703 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.232339 1539 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:52.149977 1638 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.235735 1505 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:52.264423 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.242319 1620 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:52.266330 29458 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:52.270087 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:52.308203 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 63191.1us
I20260812 19:07:52.244755 1481 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:52.240978 30670 rpcz_store.cc:276] 0812 19:07:51.217818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:51.218974 (+ 1156us) service_pool.cc:227] Handling call
0812 19:07:51.221372 (+ 2398us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:51.253505 (+ 32133us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:51.253539 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:51.745906 (+492367us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b1800291880. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ebff96 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a
0812 19:07:51.871283 (+125377us) spinlock_profiling.cc:243] Waited 4.45 ms on lock 0x7b2c000f7778. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:51.883248 (+ 11965us) spinlock_profiling.cc:243] Waited 6.44 ms on lock 0x7b2c000f7778. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:51.883731 (+ 483us) log.cc:842] Serialized 52 byte log entry
0812 19:07:52.096820 (+213089us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:52.098585 (+ 1765us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:52.114364 (+ 15779us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:52.240309 (+125945us) raft_consensus.cc:1671] finished
0812 19:07:52.240334 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:52.240513 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26556,"raft.queue_time_us":34127,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4771,"spinlock_wait_cycles":29399424,"thread_start_us":194595,"threads_started":1,"wal-append.queue_time_us":289511}
I20260812 19:07:52.245424 1525 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:52.241016 1478 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
W20260812 19:07:52.336375 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 89996.3us
I20260812 19:07:52.258033 1293 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:52.339139 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 125851us
I20260812 19:07:52.349839 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:52.315809 1330 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:52.247215 709 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.258751 1463 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:07:52.339622 1505 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:52.265679 1673 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260812 19:07:52.263109 1539 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:52.269227 1497 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:07:52.412578 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 142591us
I20260812 19:07:52.360390 1626 consensus_queue.cc:237] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.349028 1620 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.306228 1140 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.424535 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.425654 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.338725 1481 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:52.434981 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.451573 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.452721 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.460099 1293 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:52.450568 1478 raft_consensus.cc:686] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:52.461659 1478 consensus_queue.cc:237] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.462306 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.463078 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 745) took 1739 ms. Trace:
I20260812 19:07:52.463482 29338 rpcz_store.cc:276] 0812 19:07:50.723569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.724270 (+ 701us) service_pool.cc:227] Handling call
0812 19:07:52.462908 (+1738638us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003778d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:52.463024 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3441793280}
I20260812 19:07:52.464041 1505 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:52.463245 29452 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260812 19:07:52.464596 1704 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260812 19:07:52.464751 1505 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:52.467720 1515 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:52.468222 1515 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:52.468446 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:52.469064 29452 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:52.471184 29335 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:52.420416 1539 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:07:52.463841 1691 log.cc:924] Time spent T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.229s user 0.000s sys 0.000s
I20260812 19:07:52.459981 1140 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:52.468648 1515 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.427678 1513 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260812 19:07:52.425264 1495 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.526314 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.516672 1293 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:52.587035 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:52.587631 30685 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 LEARNER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260812 19:07:52.536280 1706 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.108s user 0.000s sys 0.000s
I20260812 19:07:52.463348 1557 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:07:52.426285 1507 consensus_queue.cc:1046] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.156s
I20260812 19:07:52.471163 1481 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:52.587685 30670 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:52.595499 29281 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:52.475430 1396 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.119s
I20260812 19:07:52.564050 1526 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.599799 1617 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:52.661221 1617 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:52.661430 29457 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260812 19:07:52.662231 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:52.625919 1716 log.cc:924] Time spent T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
W20260812 19:07:52.626298 1455 consensus_meta.cc:287] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.528s user 0.003s sys 0.006s
I20260812 19:07:52.599637 709 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.631497 1515 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:52.594758 1293 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.673651 1617 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:52.681509 1293 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:52.634776 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:52.682955 29456 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260812 19:07:52.683225 29452 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
W20260812 19:07:52.635372 1541 consensus_meta.cc:287] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.552s user 0.007s sys 0.003s
I20260812 19:07:52.639721 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:52.684262 30685 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:52.599233 706 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260812 19:07:52.599601 1709 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260812 19:07:52.658870 1620 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:52.660745 1692 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.423s user 0.000s sys 0.001s
I20260812 19:07:52.642297 29450 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260812 19:07:52.664902 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:52.674397 30681 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:52.690908 1526 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.691507 30670 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:52.693113 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:52.691198 29450 pending_rounds.cc:76] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 6
I20260812 19:07:52.700220 29450 pending_rounds.cc:97] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260812 19:07:52.700464 29450 raft_consensus.cc:2866] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:52.707286 29281 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:52.711714 29334 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
W20260812 19:07:52.697814 1686 log.cc:924] Time spent T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.357s user 0.001s sys 0.000s
I20260812 19:07:52.712796 1025 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:52.712055 1515 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:52.758667 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260812 19:07:52.751603 1544 consensus_meta.cc:287] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.722s user 0.012s sys 0.000s
I20260812 19:07:52.709015 1293 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260812 19:07:52.780215 30685 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:52.762435 1025 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:52.786654 1539 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.788812 1539 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:52.789501 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:52.795117 30685 raft_consensus.cc:1261] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260812 19:07:52.756611 1515 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:52.817468 30676 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
W20260812 19:07:52.750756 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 280181us
W20260812 19:07:52.797883 1537 consensus_meta.cc:287] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.603s user 0.010s sys 0.000s
I20260812 19:07:52.833041 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.797477 1025 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:52.834532 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:52.785172 1525 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.833102 1622 raft_consensus.cc:3032] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:52.818657 1515 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:52.841641 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:52.842511 29578 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260812 19:07:52.846261 29578 pending_rounds.cc:76] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting all ops after (but not including) 4
I20260812 19:07:52.846520 29578 pending_rounds.cc:97] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260812 19:07:52.846747 29578 raft_consensus.cc:2866] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:07:52.841559 29576 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0"
dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:57270
I20260812 19:07:52.842886 29567 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260812 19:07:52.836548 1537 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:52.849668 1537 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:52.850368 1525 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:52.854996 1618 consensus_meta.cc:287] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.591s user 0.009s sys 0.000s
I20260812 19:07:52.857605 30686 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:52.858536 30675 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:52.822471 1632 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260812 19:07:52.858824 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:52.913833 29580 raft_consensus.cc:2396] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:52.859373 29576 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 19:07:52.914345 29576 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:52.861619 1622 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:52.864094 30623 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:52.878575 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 424330us
I20260812 19:07:52.937777 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.937703 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.903018 29403 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.915987 29567 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260812 19:07:52.939102 29450 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:52.938503 29451 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:52.940891 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 76971.1us
I20260812 19:07:52.923657 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:52.925035 29569 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260812 19:07:52.925176 1638 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:52.927886 1649 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:52.933493 29576 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:52.955259 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:52.955876 29456 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:52.950145 29450 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:52.957051 29567 raft_consensus.cc:1226] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:52.968039 29567 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:52.968029 30625 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:52.934732 1525 raft_consensus.cc:2741] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260812 19:07:52.967097 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:52.984935 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 163886us
I20260812 19:07:52.983914 29451 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:52.987473 29569 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 7. Ops: []
I20260812 19:07:52.951145 1448 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:52.958465 1537 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:07:53.013201 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:53.013453 1732 consensus_queue.cc:1046] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:53.013692 29528 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:53.013990 29569 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:52.986629 1463 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260812 19:07:53.000015 1534 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
W20260812 19:07:53.032876 1693 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.622s user 0.000s sys 0.004s
I20260812 19:07:53.017594 1160 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260812 19:07:53.035069 1481 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.035468 1684 log.cc:924] Time spent T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.873s user 0.006s sys 0.008s
I20260812 19:07:53.026160 1717 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.028690 30675 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:53.010898 1396 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.043040 29528 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.043853 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:53.041347 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.525s user 0.017s sys 0.032s
I20260812 19:07:53.016090 1639 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260812 19:07:53.014918 1448 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:53.013317 1525 consensus_queue.cc:1046] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260812 19:07:53.067250 29456 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:53.042327 1722 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:07:53.082628 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s user 0.017s sys 0.032s
I20260812 19:07:53.068432 1726 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260812 19:07:53.092733 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 639772us
I20260812 19:07:53.007442 1622 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.098883 30629 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:53.043215 1025 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.122012 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 79056us
I20260812 19:07:53.126689 29528 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:53.085013 1293 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:53.085407 1557 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.139648 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2135) took 2198 ms. Trace:
I20260812 19:07:53.147465 29335 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:07:53.148059 29579 rpcz_store.cc:276] 0812 19:07:50.940948 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.958113 (+ 17165us) service_pool.cc:227] Handling call
0812 19:07:51.857465 (+899352us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:51.857651 (+ 186us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:51.859125 (+ 1474us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:51.859151 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:51.859591 (+ 440us) log.cc:842] Serialized 26 byte log entry
0812 19:07:52.078535 (+218944us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:52.078572 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:52.078636 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:53.034800 (+956164us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:53.139431 (+104631us) raft_consensus.cc:1671] finished
0812 19:07:53.139453 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:53.139583 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1499,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":28248192,"thread_start_us":218631,"threads_started":1,"wal-append.queue_time_us":272766}
I20260812 19:07:53.143060 29528 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.163175 29456 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:53.163892 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.164934 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:53.165557 29458 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:53.085794 1396 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.172966 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:53.170619 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 194267us
I20260812 19:07:53.127877 1722 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:53.173035 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.081709 1717 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.158097 998 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:53.185928 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.166548 1732 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:53.169529 29451 raft_consensus.cc:2354] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:53.127847 1741 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:53.174880 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:53.174046 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:53.183101 29528 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:53.183452 29446 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:53.175310 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:53.191735 29445 raft_consensus.cc:2354] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8.
I20260812 19:07:53.099530 1726 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.189405 29459 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
W20260812 19:07:53.193570 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 224989us
W20260812 19:07:53.190899 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 222930us
I20260812 19:07:53.193549 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:53.197723 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:53.195595 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:53.139950 1293 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:53.201134 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
W20260812 19:07:53.202014 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 223120us
I20260812 19:07:53.186771 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.205591 29457 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:53.209455 30625 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:53.159680 1557 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.200330 1726 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:53.200701 998 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:53.217469 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.202304 1293 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:53.221881 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:53.174628 1617 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.211516 1732 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:53.175406 1448 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.220227 998 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.250192 998 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:53.252557 1762 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:53.252501 1623 consensus_queue.cc:1035] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:07:53.253175 1762 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:53.242808 1717 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:53.243803 1160 consensus_queue.cc:1046] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:53.254104 1762 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.254827 1160 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:53.255226 1160 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:07:53.255717 1724 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.172s user 0.000s sys 0.000s
I20260812 19:07:53.256091 1160 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.245276 1396 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.245404 706 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:53.246047 1140 consensus_queue.cc:1046] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:53.214502 1463 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260812 19:07:53.215029 1741 consensus_queue.cc:1046] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260812 19:07:53.228274 1470 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260812 19:07:53.278524 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:53.279280 29579 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:53.236694 1735 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:07:53.247978 1481 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.248732 1025 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.249222 1617 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.239560 1458 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.223850 1537 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:07:53.214870 1638 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260812 19:07:53.282588 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:53.247548 1557 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:53.287568 1745 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.155s user 0.001s sys 0.000s
I20260812 19:07:53.217321 1605 consensus_queue.cc:1046] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260812 19:07:53.238862 1362 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260812 19:07:53.288841 29575 raft_consensus.cc:1261] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:07:53.289404 1160 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:07:53.289831 1748 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260812 19:07:53.290722 1632 raft_consensus.cc:3032] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:53.290978 1551 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.295926 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:53.293545 1732 raft_consensus.cc:3032] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:53.305122 1458 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.311671 1537 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:53.321201 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 103798us
W20260812 19:07:53.304368 1747 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.090s user 0.000s sys 0.001s
I20260812 19:07:53.300392 1717 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:53.303275 998 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.365259 29528 leader_election.cc:398] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:53.323915 1557 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:53.376911 29528 leader_election.cc:401] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:07:53.328238 1632 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:53.386617 30685 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:53.388362 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:53.330107 1732 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:07:53.391403 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 160045us
I20260812 19:07:53.369395 1623 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.367942 1759 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.418782 1739 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260812 19:07:53.419229 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 54120.9us
I20260812 19:07:53.437377 29528 leader_election.cc:398] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8.
W20260812 19:07:53.384776 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 174192us
I20260812 19:07:53.425577 1481 raft_consensus.cc:3032] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:53.428764 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 881) took 1533 ms. Trace:
I20260812 19:07:53.439167 30681 rpcz_store.cc:276] 0812 19:07:51.894922 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:51.982143 (+ 87221us) service_pool.cc:227] Handling call
0812 19:07:51.982413 (+ 270us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:52.691392 (+708979us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:53.428500 (+737108us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540073df54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:53.428699 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1591022464}
I20260812 19:07:53.275264 1722 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.368230 1396 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:53.380657 1557 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:53.331718 1463 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
W20260812 19:07:53.389452 1743 log.cc:924] Time spent T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.206s user 0.001s sys 0.000s
I20260812 19:07:53.389783 1620 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:53.332806 1551 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.442309 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:53.449524 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:53.448688 29530 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.470362 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:53.466893 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:53.381399 1717 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.472716 29458 raft_consensus.cc:1261] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260812 19:07:53.480271 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.481709 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.482349 29446 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:53.482836 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:53.483529 30684 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:53.441787 1639 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.441612 1557 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:53.493172 29528 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.493322 29572 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:53.462667 1735 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:07:53.469420 30675 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:53.461915 1771 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.440543 1623 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.445742 1481 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:53.511451 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:53.468612 1767 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.438112 1756 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260812 19:07:53.486543 1772 consensus_queue.cc:1046] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260812 19:07:53.461038 1632 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:53.507254 1754 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.264s user 0.001s sys 0.000s
I20260812 19:07:53.446502 1396 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.523885 29576 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:53.527338 29326 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:53.523474 30684 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:53.486572 1502 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader election won for term 2
W20260812 19:07:53.516010 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 47159.6us
I20260812 19:07:53.554873 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2291) took 2948 ms. Trace:
I20260812 19:07:53.564345 29331 rpcz_store.cc:276] 0812 19:07:50.577055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:50.577894 (+ 839us) service_pool.cc:227] Handling call
0812 19:07:51.867133 (+1289239us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:51.869449 (+ 2316us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:51.869478 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:51.869984 (+ 506us) log.cc:842] Serialized 52 byte log entry
0812 19:07:52.078223 (+208239us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:52.078265 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:52.078340 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:52.697190 (+618850us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:52.831410 (+134220us) raft_consensus.cc:1671] finished
0812 19:07:52.831428 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:53.524875 (+693447us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54003d7f50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:53.524992 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1947,"raft.queue_time_us":2881,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":3001918592,"thread_start_us":207845,"threads_started":1,"wal-append.queue_time_us":356919}
I20260812 19:07:53.476902 1739 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:07:53.467455 1443 consensus_queue.cc:1046] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
W20260812 19:07:53.583886 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 90799.4us
I20260812 19:07:53.584796 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.547747 1762 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
W20260812 19:07:53.570909 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 100597us
I20260812 19:07:53.574635 29446 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:53.555315 1741 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:53.509202 1639 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.442416 1759 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.580947 1617 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:53.621970 29281 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:53.476766 1537 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.588812 1270 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.696906 29331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:07:53.697588 29331 raft_consensus.cc:595] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:07:53.469244 1732 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.702793 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 111009us
I20260812 19:07:53.713338 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:53.723461 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:53.631465 1739 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.624792 1736 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260812 19:07:53.695890 1764 log.cc:924] Time spent T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.302s user 0.000s sys 0.001s
I20260812 19:07:53.639465 1741 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:07:53.707444 1515 consensus_meta.cc:287] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.794s user 0.013s sys 0.003s
I20260812 19:07:53.731451 29326 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:53.636734 998 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.738168 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:53.739805 29282 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:53.525213 1771 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.702664 1393 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.722810 1638 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:53.555897 1767 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.623201 1481 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.704933 1617 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.748483 1801 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.749143 1732 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:53.741104 1739 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.780963 1775 consensus_queue.cc:1035] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260812 19:07:53.781383 1710 log.cc:924] Time spent T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 1.086s user 0.000s sys 0.007s
W20260812 19:07:53.781980 1785 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.219s user 0.001s sys 0.000s
I20260812 19:07:53.749965 1515 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.782778 1802 log.cc:924] Time spent T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260812 19:07:53.761579 1393 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.784029 1797 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.780998 1638 raft_consensus.cc:686] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:07:53.835593 1746 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.624s user 0.001s sys 0.001s
I20260812 19:07:53.836364 1771 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:53.836021 1638 consensus_queue.cc:237] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.837594 1759 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:53.784070 1772 raft_consensus.cc:3032] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:53.838126 1772 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:53.838723 1623 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.785915 1790 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:53.839970 1639 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.706400 1270 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.841292 1767 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.842127 1270 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.762728 1717 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:53.842461 1767 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.842887 1717 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.771293 1699 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260812 19:07:53.843434 1717 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.844156 1393 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.817098 1806 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.844837 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 734) took 1669 ms. Trace:
I20260812 19:07:53.845371 1717 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:53.777719 1761 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.392s user 0.000s sys 0.001s
W20260812 19:07:53.799480 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 54194.1us
W20260812 19:07:53.791712 1768 log.cc:924] Time spent T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.269s user 0.001s sys 0.000s
I20260812 19:07:53.824951 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:53.847646 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 696) took 1422 ms. Trace:
I20260812 19:07:53.799458 1362 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:07:53.849298 29530 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:53.845207 29453 rpcz_store.cc:276] 0812 19:07:52.175409 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:52.177079 (+ 1670us) service_pool.cc:227] Handling call
0812 19:07:52.177432 (+ 353us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:52.177780 (+ 348us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:52.177807 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:52.201944 (+ 24137us) log.cc:842] Serialized 26 byte log entry
0812 19:07:52.307293 (+105349us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:52.351476 (+ 44183us) log.cc:842] Serialized 14 byte log entry
0812 19:07:52.351691 (+ 215us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:52.351761 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:52.695180 (+343419us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b180043cae0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ef2638 00007f9c66ef5b11 00007f9c66ef5ac9 00007f9c66ef5a59 00007f9c66ef5a21 00007f9c66ef4d1d 00007f9c7014a204 00007f9c7014a039 00007f9c66ed4dc6
0812 19:07:53.723010 (+1027830us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:53.844647 (+121637us) raft_consensus.cc:1671] finished
0812 19:07:53.844667 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:53.844784 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":43989,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":979,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":34009,"spinlock_wait_cycles":34282112,"thread_start_us":65106,"threads_started":1,"wal-append.queue_time_us":209421}
I20260812 19:07:53.839118 1739 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.839638 1790 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.844991 1806 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.786334 1781 log.cc:924] Time spent T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.153s user 0.001s sys 0.000s
W20260812 19:07:53.860644 1786 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.161s user 0.000s sys 0.001s
I20260812 19:07:53.790084 1551 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:53.836603 1515 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.861982 1806 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:53.866353 1160 consensus_meta.cc:287] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.538s user 0.009s sys 0.000s
I20260812 19:07:53.852249 1717 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:53.867702 1717 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.869472 1617 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.869630 1717 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:53.798529 1801 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:53.870584 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 827) took 1176 ms. Trace:
I20260812 19:07:53.871726 1801 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:53.857554 1778 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.236s user 0.000s sys 0.001s
I20260812 19:07:53.843022 1767 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.870975 30670 rpcz_store.cc:276] 0812 19:07:52.694045 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:52.694194 (+ 149us) service_pool.cc:227] Handling call
0812 19:07:52.694711 (+ 517us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:52.695218 (+ 507us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:52.695245 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:52.696316 (+ 1071us) log.cc:842] Serialized 590 byte log entry
0812 19:07:53.080412 (+384096us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:53.080472 (+ 60us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:53.080545 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:53.830493 (+749948us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:53.849233 (+ 18740us) raft_consensus.cc:1671] finished
0812 19:07:53.849252 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:53.870513 (+ 21261us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":5444,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"spinlock_wait_cycles":4663168,"thread_start_us":381605,"threads_started":1,"wal-append.queue_time_us":462566}
I20260812 19:07:53.876915 1767 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.880192 1557 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.881403 29457 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:07:53.852741 1797 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.888375 1790 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:53.827390 1443 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:53.848016 29574 rpcz_store.cc:276] 0812 19:07:52.424931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:52.426862 (+ 1931us) service_pool.cc:227] Handling call
0812 19:07:52.460992 (+ 34130us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:52.963641 (+502649us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:52.985391 (+ 21750us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:53.011595 (+ 26204us) log.cc:842] Serialized 26 byte log entry
0812 19:07:53.224559 (+212964us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:53.231718 (+ 7159us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:53.259000 (+ 27282us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:53.847382 (+588382us) raft_consensus.cc:1671] finished
0812 19:07:53.847406 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:53.847588 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9781,"raft.queue_time_us":174018,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"thread_start_us":529968,"threads_started":2,"wal-append.queue_time_us":300398}
I20260812 19:07:53.885761 1797 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:53.900774 1793 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:53.928510 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2200) took 1087 ms. Trace:
I20260812 19:07:53.928975 29335 rpcz_store.cc:276] 0812 19:07:52.840846 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:52.840973 (+ 127us) service_pool.cc:227] Handling call
0812 19:07:52.847123 (+ 6150us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:53.157252 (+310129us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:53.928209 (+770957us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54003778d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:53.928429 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":690149760}
W20260812 19:07:53.934334 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 349602us
I20260812 19:07:53.935212 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:53.752920 1762 raft_consensus.cc:686] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:53.943599 1762 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.930414 1726 consensus_meta.cc:287] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.702s user 0.011s sys 0.000s
I20260812 19:07:53.838948 1772 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:53.952576 1293 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.723s user 0.006s sys 0.005s
I20260812 19:07:53.909636 1443 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:53.976522 1797 consensus_queue.cc:1046] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260812 19:07:54.001499 1502 raft_consensus.cc:686] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:53.994653 1605 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.056475 1605 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.057711 1617 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:54.058374 1617 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.060458 1617 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.060978 1605 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.065320 1622 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:54.031057 1443 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:54.035181 1496 raft_consensus.cc:3032] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:54.079615 1496 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:54.080065 1605 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:53.970912 1557 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.081259 1605 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.081187 1496 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.977231 1767 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:53.955722 1793 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.043702 1801 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:54.044015 1805 log.cc:924] Time spent T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.068s user 0.002s sys 0.000s
I20260812 19:07:53.977164 1812 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:53.972841 1726 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:53.972002 1810 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:53.977039 1293 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.067588 1481 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:54.085142 1767 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.090231 1557 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.109768 1557 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:54.112279 1726 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.113495 29577 raft_consensus.cc:1226] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:54.076045 1735 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.114152 1810 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.115425 1735 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:54.117717 29450 raft_consensus.cc:2379] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:54.099089 1455 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:54.134509 1455 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:54.135964 1455 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.137567 1455 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:54.108924 1507 raft_consensus.cc:3032] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:54.112984 1557 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.090322 1605 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.163221 1807 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.168136 1807 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.101457 1793 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:54.174590 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.607s user 0.010s sys 0.010s
W20260812 19:07:54.174932 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s user 0.010s sys 0.010s
I20260812 19:07:54.166318 1507 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:54.108489 1801 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:54.175805 29445 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.680s user 0.001s sys 0.018s
W20260812 19:07:54.181154 29445 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.686s user 0.001s sys 0.019s
W20260812 19:07:54.178336 30681 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s user 0.017s sys 0.012s
W20260812 19:07:54.181689 30681 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.553s user 0.017s sys 0.012s
W20260812 19:07:54.176680 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.003s sys 0.016s
W20260812 19:07:54.182296 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.582s user 0.004s sys 0.016s
W20260812 19:07:54.197289 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.572s user 0.019s sys 0.027s
W20260812 19:07:54.197738 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.572s user 0.019s sys 0.027s
I20260812 19:07:54.171841 1293 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.207791 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:54.208848 29403 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.007s sys 0.014s
W20260812 19:07:54.210177 29403 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.511s user 0.007s sys 0.014s
W20260812 19:07:54.204026 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 461729us
I20260812 19:07:54.268169 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.269373 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.280084 29578 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:54.102864 1502 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.283278 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.285001 29576 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:54.285892 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.190639 1729 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.286506 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.267091 1292 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.296996 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.299122 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:54.325976 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.326611 29332 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:54.327397 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:54.327420 30676 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
W20260812 19:07:54.299340 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 92767.8us
I20260812 19:07:54.323416 30686 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:54.324919 30685 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:07:54.270032 1025 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.337086 30685 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:54.337486 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:54.270293 1458 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.287709 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.323526 1292 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.362211 1025 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.362983 1741 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:54.365896 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.366618 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:54.298934 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.340660 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.363755 1741 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.369861 29580 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:54.370726 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.370888 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.266057 1557 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.371448 30679 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:54.371978 1557 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.373453 1292 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:54.374123 1293 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.339179 1717 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.374923 1293 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.375834 1717 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:54.380811 1292 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:54.385977 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.336585 1793 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:54.388031 1793 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.376753 1807 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.392031 1448 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:07:54.392659 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.393509 29335 raft_consensus.cc:2379] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 3: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:54.365039 1726 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:54.360792 1830 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.403061 1396 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:54.405181 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.405591 1632 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 7
I20260812 19:07:54.406375 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.408347 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.394433 30629 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:54.413389 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 46735.3us
I20260812 19:07:54.417101 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:54.322820 1729 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.376636 1557 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.287177 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.330912 1735 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:54.310678 1507 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.381584 1292 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.550949 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:54.566730 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.567934 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.462674 1807 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:54.507685 1557 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.569027 1292 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.568789 1807 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.531941 1801 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.541019 30670 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:54.571290 1539 raft_consensus.cc:3032] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Stepping down as leader of term 1
I20260812 19:07:54.571856 1539 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:54.541684 1741 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.511451 1729 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:54.538479 1293 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.575950 1507 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:54.576323 1793 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.474000 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.580776 30676 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:54.569448 1557 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.582180 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.583014 29575 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:54.584057 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.585610 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.573134 1539 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.594249 1807 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.595022 1810 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.598403 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.460201 30684 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:54.424415 29332 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:54.483748 29578 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:07:54.462224 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:54.621201 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 159164us
I20260812 19:07:54.621925 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:54.623523 29450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2417) took 1461 ms (client timeout 1591 ms). Trace:
W20260812 19:07:54.623939 29450 rpcz_store.cc:269] 0812 19:07:53.162299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.175085 (+ 12786us) service_pool.cc:227] Handling call
0812 19:07:54.623318 (+1448233us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b5400213254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:54.623461 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2128276224}
I20260812 19:07:54.624989 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.471732 1799 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:54.432307 1841 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.625777 1799 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:54.466480 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:54.626614 29446 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: [5.5-5.5]
W20260812 19:07:54.626865 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 160476us
I20260812 19:07:54.627038 1463 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:54.625656 29450 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:54.627482 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.426409 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.461871 29326 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:07:54.598436 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:54.625061 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.628942 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:54.630172 29578 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:54.604317 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:54.603206 30629 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:54.620509 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:54.633559 29575 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:54.627533 1463 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:54.636549 1463 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:54.636947 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:54.637305 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:54.638401 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:54.638815 29457 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:07:54.640102 29457 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:54.640519 29446 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 7
I20260812 19:07:54.641332 29457 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:54.642387 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.643941 29456 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:07:54.626470 1799 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:54.640111 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.651350 29446 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:07:54.650458 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 47377.5us
I20260812 19:07:54.656970 30629 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:54.433622 1821 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.423949 1726 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.504211 1025 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.431247 1829 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:54.739815 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:54.508536 1767 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.603910 1539 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 1 FOLLOWER]: Advancing to term 2
W20260812 19:07:54.733587 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 68143.8us
I20260812 19:07:54.629907 1717 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:54.664623 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.739281 30684 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
W20260812 19:07:54.667768 1537 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s user 0.019s sys 0.034s
I20260812 19:07:54.759550 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 708) took 1547 ms. Trace:
I20260812 19:07:54.658702 1810 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.626461 1841 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.787985 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2073) took 1309 ms. Trace:
W20260812 19:07:54.767236 1740 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.305s user 0.017s sys 0.046s
I20260812 19:07:54.771832 1813 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.658634 30675 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:07:54.670537 1455 consensus_meta.cc:287] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.510s user 0.013s sys 0.000s
I20260812 19:07:54.778915 1770 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:54.779667 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.780591 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:54.659296 1623 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:54.784222 29577 rpcz_store.cc:276] 0812 19:07:53.212099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.213621 (+ 1522us) service_pool.cc:227] Handling call
0812 19:07:53.219768 (+ 6147us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:54.113931 (+894163us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:54.759274 (+645343us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400135454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:54.759481 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2054955904}
I20260812 19:07:54.788389 29458 rpcz_store.cc:276] 0812 19:07:53.478258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.498077 (+ 19819us) service_pool.cc:227] Handling call
0812 19:07:53.498367 (+ 290us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:53.505769 (+ 7402us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:53.505801 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:53.505841 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:53.505872 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:54.787772 (+1281900us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b54002ff8d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:54.787817 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:54.787931 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":16458624}
I20260812 19:07:54.792057 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:54.735299 1160 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:54.738070 1557 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.469872 1830 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:55.037844 29453 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.104s user 0.013s sys 0.036s
W20260812 19:07:55.249938 29453 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.317s user 0.013s sys 0.045s
W20260812 19:07:54.949383 29459 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s user 0.035s sys 0.064s
W20260812 19:07:55.251478 29459 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.463s user 0.042s sys 0.070s
I20260812 19:07:54.931125 29451 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:55.241366 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:54.871196 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.250784 29457 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:55.260640 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:55.272698 29450 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:07:55.054075 29575 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:55.246328 29446 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:54.870255 29580 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
W20260812 19:07:54.784135 1537 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.941s user 0.020s sys 0.034s
W20260812 19:07:54.794571 1478 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.208s user 0.022s sys 0.050s
W20260812 19:07:55.281587 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 501037us
W20260812 19:07:55.344483 29338 consensus_meta.cc:287] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.532s user 0.008s sys 0.002s
I20260812 19:07:55.351922 29338 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:07:54.899715 1740 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.458s user 0.017s sys 0.047s
I20260812 19:07:54.788141 1829 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:54.791484 1767 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:55.244946 1718 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.204s user 0.009s sys 0.024s
I20260812 19:07:55.169852 1293 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:54.869661 1722 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260812 19:07:55.363943 29567 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.855s user 0.027s sys 0.080s
I20260812 19:07:55.140157 1801 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.945623 1741 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.069908 1793 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:55.359272 1762 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.295s user 0.044s sys 0.100s
W20260812 19:07:55.351861 1478 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.767s user 0.026s sys 0.065s
I20260812 19:07:55.059022 1847 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.350459 1770 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.903409 1736 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
W20260812 19:07:55.402525 29567 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.893s user 0.027s sys 0.086s
I20260812 19:07:55.401396 1557 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:55.276367 1807 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:54.869463 1717 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:55.344691 1481 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:55.355280 1605 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:55.434682 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.128315 1622 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.070089 1735 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:55.256438 1826 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:55.401106 1718 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.372s user 0.010s sys 0.025s
I20260812 19:07:55.397383 1537 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.369652 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.094310 1822 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:54.947932 1842 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:55.428434 1762 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.380s user 0.044s sys 0.104s
I20260812 19:07:55.094121 1852 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.451627 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2203) took 1116 ms. Trace:
I20260812 19:07:55.536058 29580 rpcz_store.cc:276] 0812 19:07:54.335061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.370622 (+ 35561us) service_pool.cc:227] Handling call
0812 19:07:55.451432 (+1080810us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54000afd54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:55.451563 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1146874240}
I20260812 19:07:55.402066 1829 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:55.412303 1649 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:55.279695 1160 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.473387 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.475819 30685 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260812 19:07:55.502645 1557 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260812 19:07:55.478420 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.502534 1717 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.483232 1830 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:55.483831 1841 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.484550 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.552841 30671 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:07:55.488122 1539 consensus_meta.cc:287] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.709s user 0.007s sys 0.003s
W20260812 19:07:55.492365 1507 consensus_meta.cc:287] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.916s user 0.010s sys 0.000s
W20260812 19:07:55.492728 1799 consensus_meta.cc:287] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.846s user 0.013s sys 0.000s
I20260812 19:07:55.444519 1726 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:54.932592 1813 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.809664 1810 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:54.784642 1821 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.279453 1623 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:55.446612 1736 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:55.510975 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:07:55.584340 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 73487us
I20260812 19:07:55.586017 29281 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:55.411113 1767 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.420389 1801 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.361240 1824 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.212s
I20260812 19:07:55.428033 1293 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.434945 1793 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.427894 1741 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.501652 1605 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.450444 1770 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:55.444777 1847 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.544885 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.503325 1481 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.550159 1160 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:55.551503 1853 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.150s user 0.000s sys 0.001s
I20260812 19:07:55.550207 1496 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:55.561704 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:55.696720 30676 raft_consensus.cc:2379] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:55.578382 1820 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:55.517693 1735 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.517972 1622 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.529863 1537 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.520092 1826 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.556605 1830 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:55.625983 29528 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.873s user 0.025s sys 0.109s
W20260812 19:07:55.710758 29528 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.958s user 0.028s sys 0.109s
W20260812 19:07:55.711128 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 958154us
I20260812 19:07:55.611040 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:55.583899 1726 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:55.544744 1649 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.580930 1799 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.690747 30685 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:55.620514 1775 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:07:55.589099 1623 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:07:55.590387 1463 consensus_meta.cc:287] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.953s user 0.013s sys 0.000s
I20260812 19:07:55.537267 1842 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:55.712718 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 101773us
I20260812 19:07:55.772979 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:55.648968 29576 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:55.589239 29574 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:55.543428 1822 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.751771 29579 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:55.588954 1632 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:55.552187 1852 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:55.762920 1726 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.764031 1820 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.781234 1507 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:55.719794 1830 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.876299 1767 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.877381 1830 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.812165 1603 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:55.878051 1603 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:55.878502 1603 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:55.879426 1852 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:55.774312 1775 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:55.882198 1775 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:07:55.764431 1772 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:55.887558 1799 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:55.889132 1842 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:55.770097 1160 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:55.890341 1160 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.891388 1293 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.812428 1458 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.720388 1617 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
W20260812 19:07:55.827966 1859 log.cc:924] Time spent T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
W20260812 19:07:55.913549 30681 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.585s user 0.008s sys 0.054s
W20260812 19:07:55.913944 30681 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.603s user 0.008s sys 0.054s
W20260812 19:07:55.862920 29572 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.724s user 0.014s sys 0.032s
W20260812 19:07:55.829859 29282 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.086s user 0.030s sys 0.131s
W20260812 19:07:55.915061 29572 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.776s user 0.014s sys 0.032s
W20260812 19:07:55.915249 29282 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.171s user 0.030s sys 0.132s
W20260812 19:07:55.916006 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 2.17612e+06us
W20260812 19:07:55.767020 1638 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.646s user 0.004s sys 0.039s
W20260812 19:07:55.814379 29452 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.782s user 0.016s sys 0.044s
W20260812 19:07:55.861936 1855 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.098s user 0.023s sys 0.012s
I20260812 19:07:55.811657 1025 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:55.779975 1463 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:55.814904 30686 raft_consensus.cc:2379] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:07:55.896617 29331 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.397s user 0.020s sys 0.050s
W20260812 19:07:55.908804 29445 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.597s user 0.003s sys 0.036s
I20260812 19:07:55.874683 1813 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:55.950011 29452 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.918s user 0.017s sys 0.044s
I20260812 19:07:55.893159 1717 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:56.012159 29331 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.513s user 0.024s sys 0.061s
W20260812 19:07:56.012593 1854 log.cc:924] Time spent T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.189s user 0.000s sys 0.000s
I20260812 19:07:55.917425 1539 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:55.953554 1793 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:56.013327 1862 log.cc:924] Time spent T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.200s user 0.000s sys 0.000s
I20260812 19:07:55.914021 1458 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:55.951409 1855 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.188s user 0.023s sys 0.012s
I20260812 19:07:56.032480 1735 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.952286 29282 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:56.014899 1826 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:56.015122 1623 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.018406 1845 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.249s user 0.000s sys 0.000s
I20260812 19:07:56.030224 1813 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:55.949851 1638 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.829s user 0.008s sys 0.039s
I20260812 19:07:56.029184 1852 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:56.014134 1847 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:55.974058 1448 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.497s user 0.015s sys 0.051s
W20260812 19:07:56.122225 1448 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.664s user 0.015s sys 0.058s
I20260812 19:07:56.122567 1463 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:56.062640 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 110432us
I20260812 19:07:56.029326 1806 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:56.127211 29576 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2160) took 1842 ms (client timeout 2029 ms). Trace:
W20260812 19:07:56.127656 29576 rpcz_store.cc:269] 0812 19:07:54.284483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.567688 (+283205us) service_pool.cc:227] Handling call
0812 19:07:56.127016 (+1559328us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400388e54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:56.127143 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2484605696}
I20260812 19:07:56.131217 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2155) took 1851 ms. Trace:
I20260812 19:07:56.131691 29579 rpcz_store.cc:276] 0812 19:07:54.280242 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.280392 (+ 150us) service_pool.cc:227] Handling call
0812 19:07:54.791864 (+511472us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:56.131056 (+1339192us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540012be54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:56.131163 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1132670080}
I20260812 19:07:56.034624 1801 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:56.022320 29445 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.724s user 0.004s sys 0.048s
I20260812 19:07:56.029838 1481 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:56.036285 1741 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.036134 1810 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:56.081364 1537 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:56.118603 1852 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.032359 1539 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:56.177865 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.895s user 0.009s sys 0.047s
I20260812 19:07:56.131256 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2207) took 1665 ms. Trace:
I20260812 19:07:56.142942 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2208) took 1511 ms. Trace:
I20260812 19:07:56.102571 1649 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:56.161027 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.880s user 0.050s sys 0.076s
W20260812 19:07:56.179131 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.898s user 0.050s sys 0.077s
I20260812 19:07:56.178577 29580 rpcz_store.cc:276] 0812 19:07:54.465593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.544025 (+1078432us) service_pool.cc:227] Handling call
0812 19:07:55.573231 (+ 29206us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:55.648207 (+ 74976us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:55.648235 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:55.648275 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:55.648303 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.131074 (+482771us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b5400172354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:56.131112 (+ 38us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:56.131207 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":171916160}
I20260812 19:07:56.122560 1726 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:55.969552 1821 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:56.063905 1478 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260812 19:07:56.178540 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.896s user 0.009s sys 0.048s
I20260812 19:07:55.898756 1507 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.198086 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.844s user 0.027s sys 0.030s
I20260812 19:07:56.198256 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2082) took 2709 ms. Trace:
W20260812 19:07:56.199189 30686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 900) took 1776 ms (client timeout 1868 ms). Trace:
W20260812 19:07:56.239653 30686 rpcz_store.cc:269] 0812 19:07:54.422996 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.423211 (+ 215us) service_pool.cc:227] Handling call
0812 19:07:56.198946 (+1775735us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400677d54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:56.199126 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2725598976}
I20260812 19:07:56.178918 29576 rpcz_store.cc:276] 0812 19:07:54.631272 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.142080 (+1510808us) service_pool.cc:227] Handling call
0812 19:07:56.142432 (+ 352us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:56.142633 (+ 201us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:56.142656 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:56.142695 (+ 39us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:56.142726 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.142775 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:56.142883 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:56.218771 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 936130us
I20260812 19:07:55.968644 1617 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:56.103796 1813 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:56.223224 29403 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.964s user 0.006s sys 0.055s
I20260812 19:07:56.158970 1822 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:56.235713 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 112123us
W20260812 19:07:56.241454 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.888s user 0.028s sys 0.030s
I20260812 19:07:56.199892 1741 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:56.134771 1806 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:56.238481 29459 rpcz_store.cc:276] 0812 19:07:53.488875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.497222 (+ 8347us) service_pool.cc:227] Handling call
0812 19:07:53.497778 (+ 556us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:53.788267 (+290489us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:53.788296 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:53.788705 (+ 409us) log.cc:842] Serialized 26 byte log entry
0812 19:07:55.251887 (+1463182us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:55.251926 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:55.688371 (+436445us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54003bcc54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:55.688419 (+ 48us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:56.163928 (+475509us) raft_consensus.cc:1671] finished
0812 19:07:56.163952 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:56.198180 (+ 34228us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":74290,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"spinlock_wait_cycles":667189632,"thread_start_us":1462901,"threads_started":1,"wal-append.queue_time_us":1906864}
W20260812 19:07:56.280579 29403 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.022s user 0.007s sys 0.056s
W20260812 19:07:56.281167 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 1.0224e+06us
W20260812 19:07:56.282122 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 928578us
I20260812 19:07:56.282855 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:56.195430 1470 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.587s user 0.025s sys 0.063s
I20260812 19:07:56.226281 1160 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.287459 1822 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:07:56.236963 1821 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:56.331480 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 48705.6us
I20260812 19:07:56.244426 1726 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:56.029247 1025 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.284119 1801 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:56.370234 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 83395.2us
W20260812 19:07:56.293990 1470 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.694s user 0.028s sys 0.064s
W20260812 19:07:56.376955 29575 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.965s user 0.005s sys 0.034s
W20260812 19:07:56.406757 29575 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.995s user 0.005s sys 0.036s
I20260812 19:07:56.393385 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:56.277542 1478 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:56.403985 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:56.404788 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:56.410375 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:56.413668 29450 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:56.414721 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:56.413175 29446 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:56.417073 29450 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
W20260812 19:07:56.413218 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.037s user 0.016s sys 0.058s
I20260812 19:07:56.285629 1839 consensus_queue.cc:1046] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
W20260812 19:07:56.550961 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.175s user 0.026s sys 0.067s
I20260812 19:07:56.393868 1822 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:56.377449 1847 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:56.619047 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 116946us
I20260812 19:07:56.289067 1741 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.472561 1622 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:56.407207 1293 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:07:56.706867 29403 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 2.029s (SENT) [suppressed 15 similar messages]
I20260812 19:07:56.419643 1458 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:56.398769 1448 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:56.628422 1820 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.637136 1822 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:56.637998 1623 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.407259 1717 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:56.743593 1603 consensus_meta.cc:287] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.865s user 0.006s sys 0.009s
I20260812 19:07:56.744234 1603 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.744141 1717 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.746702 1861 log.cc:924] Time spent T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.348s user 0.000s sys 0.000s
I20260812 19:07:56.671830 1839 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:56.474478 1478 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.407439 1632 consensus_queue.cc:1046] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.329s
I20260812 19:07:56.765683 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:56.720817 1810 raft_consensus.cc:3032] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
W20260812 19:07:56.763746 30626 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.228s user 0.017s sys 0.066s
I20260812 19:07:56.763731 1603 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.804440 1810 raft_consensus.cc:729] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:56.805124 1863 log.cc:924] Time spent T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.427s user 0.000s sys 0.000s
I20260812 19:07:56.764528 1852 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:56.804312 30626 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.276s user 0.017s sys 0.066s
W20260812 19:07:56.807291 1496 consensus_meta.cc:287] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.104s user 0.006s sys 0.013s
I20260812 19:07:56.807823 29569 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
W20260812 19:07:56.809047 1772 consensus_meta.cc:287] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.923s user 0.000s sys 0.011s
I20260812 19:07:56.809541 1455 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:56.810042 1455 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.795401 29527 connection.cc:232] RPC call timeout handler was delayed by -1.1508373848628253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260812 19:07:56.807186 29338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2319) took 3084 ms (client timeout 1699 ms). Trace:
W20260812 19:07:56.813370 29338 rpcz_store.cc:269] 0812 19:07:53.723009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.723178 (+ 169us) service_pool.cc:227] Handling call
0812 19:07:56.806978 (+3083800us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400092ed4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:56.807094 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2502689024}
I20260812 19:07:56.399123 1726 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:56.408919 1801 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.803592 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2084) took 3303 ms. Trace:
I20260812 19:07:56.794214 29450 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:56.832089 29450 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:56.639326 1847 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.822769 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:56.826663 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:07:56.837018 30629 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.558s user 0.025s sys 0.024s
I20260812 19:07:56.823436 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:56.409777 1821 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.830667 30684 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.282s user 0.017s sys 0.034s
W20260812 19:07:56.827898 29578 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.074s user 0.021s sys 0.028s
I20260812 19:07:56.835110 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:56.623481 1617 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.830435 29452 rpcz_store.cc:276] 0812 19:07:53.499721 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.559421 (+ 59700us) service_pool.cc:227] Handling call
0812 19:07:53.618226 (+ 58805us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:53.971283 (+353057us) raft_consensus.cc:1520] Early marking committed up to index 4
0812 19:07:54.023182 (+ 51899us) log.cc:842] Serialized 14 byte log entry
0812 19:07:56.023539 (+2000357us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:56.025593 (+ 2054us) log.cc:842] Serialized 26 byte log entry
0812 19:07:56.025740 (+ 147us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:56.025770 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.025840 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:56.803364 (+777524us) raft_consensus.cc:1671] finished
0812 19:07:56.803384 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:56.803520 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":45774,"prepare.run_cpu_time_us":4861,"prepare.run_wall_time_us":9974,"raft.queue_time_us":202804,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":32537,"thread_start_us":2321857,"threads_started":2,"wal-append.queue_time_us":2211882}
I20260812 19:07:56.801858 1622 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:56.878265 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.605s user 0.010s sys 0.044s
W20260812 19:07:56.878813 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.606s user 0.010s sys 0.044s
I20260812 19:07:56.740763 1448 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.840050 30684 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.291s user 0.017s sys 0.034s
W20260812 19:07:56.843276 30629 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.564s user 0.025s sys 0.024s
I20260812 19:07:56.844307 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:56.841391 29578 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.087s user 0.022s sys 0.029s
I20260812 19:07:56.738207 1293 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.822300 1160 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:56.821460 1741 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:56.766561 1839 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:56.805930 1852 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260812 19:07:56.805328 1810 consensus_queue.cc:260] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:56.881407 29567 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.439s user 0.035s sys 0.069s
I20260812 19:07:57.017849 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 838) took 3538 ms. Trace:
W20260812 19:07:57.183732 30625 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.369s user 0.030s sys 0.078s
W20260812 19:07:57.187095 30625 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.373s user 0.030s sys 0.078s
W20260812 19:07:57.187372 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 1.40063e+06us
W20260812 19:07:56.845489 1515 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.291s user 0.022s sys 0.027s
I20260812 19:07:56.838493 1799 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:07:56.813124 1807 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:57.185071 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 853) took 1709 ms. Trace:
I20260812 19:07:57.204689 30684 rpcz_store.cc:276] 0812 19:07:55.475311 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.544594 (+ 69283us) service_pool.cc:227] Handling call
0812 19:07:55.548438 (+ 3844us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:57.184714 (+1636276us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:57.184742 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.184780 (+ 38us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:57.184816 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.184894 (+ 78us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.185013 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":171565,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":14575,"thread_start_us":1635850,"threads_started":1}
I20260812 19:07:56.828363 1458 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:57.190544 30681 rpcz_store.cc:276] 0812 19:07:53.479415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.479554 (+ 139us) service_pool.cc:227] Handling call
0812 19:07:53.507456 (+ 27902us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:53.580187 (+ 72731us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:54.182082 (+601895us) log.cc:842] Serialized 14 byte log entry
0812 19:07:55.914384 (+1732302us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:55.930860 (+ 16476us) log.cc:842] Serialized 26 byte log entry
0812 19:07:55.931034 (+ 174us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:55.931067 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:55.931137 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:57.017609 (+1086472us) raft_consensus.cc:1671] finished
0812 19:07:57.017638 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.017784 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1000802,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":220336,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":52867,"thread_start_us":2228486,"threads_started":3,"wal-append.queue_time_us":2776927}
I20260812 19:07:56.831022 1772 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:57.016611 1718 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.476s user 0.019s sys 0.041s
I20260812 19:07:57.072252 1025 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:56.845659 29453 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.595s user 0.026s sys 0.024s
W20260812 19:07:57.565265 29453 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.315s user 0.029s sys 0.025s
I20260812 19:07:57.566589 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2085) took 3634 ms. Trace:
I20260812 19:07:57.567003 29453 rpcz_store.cc:276] 0812 19:07:53.932085 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.932241 (+ 156us) service_pool.cc:227] Handling call
0812 19:07:53.932589 (+ 348us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:53.932805 (+ 216us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:53.933146 (+ 341us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.565867 (+3632721us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.565998 (+ 131us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:57.566135 (+ 137us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.566271 (+ 136us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.566458 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":275,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":75,"raft.queue_time_us":29019,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":63449,"spinlock_wait_cycles":101120,"thread_start_us":3632134,"threads_started":2,"wal-append.queue_time_us":1769003,"wal-append.run_cpu_time_us":12648,"wal-append.run_wall_time_us":1246733}
I20260812 19:07:57.521279 29450 raft_consensus.cc:595] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:57.569648 29450 raft_consensus.cc:603] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260812 19:07:57.148010 29528 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.425s user 0.015s sys 0.051s
I20260812 19:07:57.521477 29451 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:07:57.570700 29528 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.848s user 0.015s sys 0.054s
I20260812 19:07:57.535945 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:57.574566 29335 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:57.538918 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.540503 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.578596 29326 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:57.540858 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.541347 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:57.577630 29528 connection.cc:232] RPC call timeout handler was delayed by -1.8550884178839624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260812 19:07:57.585695 29528 connection.cc:232] RPC call timeout handler was delayed by -1.6256936579011381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
W20260812 19:07:57.586113 29528 connection.cc:232] RPC call timeout handler was delayed by -1.300118796993047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2069s for a response
W20260812 19:07:57.586407 29528 connection.cc:232] RPC call timeout handler was delayed by -1.2559280579444021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
I20260812 19:07:57.541731 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:57.183276 29567 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.741s user 0.039s sys 0.077s
W20260812 19:07:57.546447 29575 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.138s user 0.000s sys 0.012s
W20260812 19:07:57.546842 29579 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.367s user 0.007s sys 0.010s
W20260812 19:07:57.547411 29580 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.135s user 0.002s sys 0.010s
I20260812 19:07:56.843858 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:57.562638 30685 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.741s user 0.010s sys 0.005s
W20260812 19:07:57.199519 1515 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.649s user 0.022s sys 0.032s
I20260812 19:07:57.245761 1605 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:57.242017 1718 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.704s user 0.025s sys 0.046s
I20260812 19:07:57.672493 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2205) took 3213 ms. Trace:
I20260812 19:07:57.672962 29577 rpcz_store.cc:276] 0812 19:07:54.459255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.324262 (+865007us) service_pool.cc:227] Handling call
0812 19:07:56.128539 (+804277us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:56.128736 (+ 197us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:56.128762 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:56.128800 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:56.128836 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.128883 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.672049 (+1543166us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400388e50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:57.672271 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1764745600}
I20260812 19:07:57.673857 1889 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:57.005479 1717 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.255880 1726 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:57.356148 1736 consensus_meta.cc:287] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.705s user 0.013s sys 0.000s
I20260812 19:07:57.243698 1722 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:57.383597 1770 consensus_meta.cc:287] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.677s user 0.016s sys 0.005s
I20260812 19:07:57.243147 1772 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.690598 1293 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.333688 1775 consensus_meta.cc:287] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 1.451s user 0.009s sys 0.005s
I20260812 19:07:57.698091 1775 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.698997 1770 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.701049 1775 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.209100 1799 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.702215 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 710) took 4301 ms. Trace:
I20260812 19:07:57.702673 29567 rpcz_store.cc:276] 0812 19:07:53.390569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.390737 (+ 168us) service_pool.cc:227] Handling call
0812 19:07:53.391065 (+ 328us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:53.391271 (+ 206us) raft_consensus.cc:1520] Early marking committed up to index 5
0812 19:07:53.509004 (+117733us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.677172 (+4168168us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.677293 (+ 121us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:57.682180 (+ 4887us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.686734 (+ 4554us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.692359 (+ 5625us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44328,"prepare.queue_time_us":126583,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":254183,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7874,"thread_start_us":4189188,"threads_started":3,"wal-append.queue_time_us":2376398,"wal-append.run_cpu_time_us":1367,"wal-append.run_wall_time_us":1004288}
I20260812 19:07:57.704530 1799 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.215333 1801 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.209146 1807 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:56.894881 1813 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.321026 1821 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.416584 1822 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.654s user 0.008s sys 0.002s
I20260812 19:07:57.714972 1932 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:57.715698 1822 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:07:57.716184 1932 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:57.331952 1829 consensus_meta.cc:287] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.784s user 0.013s sys 0.010s
I20260812 19:07:57.329465 1839 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.259323 1847 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.271396 1852 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:07:57.251545 1864 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.526s user 0.000s sys 0.001s
I20260812 19:07:57.696404 1557 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:57.719431 29458 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [3.4-5.5]
W20260812 19:07:57.356798 1868 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.314s user 0.001s sys 0.001s
I20260812 19:07:57.721391 1557 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.723273 1822 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:57.315133 1875 log.cc:924] Time spent T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.488s user 0.000s sys 0.001s
I20260812 19:07:57.725126 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:57.725224 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2078) took 4245 ms. Trace:
I20260812 19:07:57.725253 1557 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.267328 1886 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:57.726198 1886 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:57.726684 1886 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:57.727078 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 771) took 3335 ms. Trace:
I20260812 19:07:57.726840 1852 raft_consensus.cc:686] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:57.322880 1899 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:57.727607 29331 rpcz_store.cc:276] 0812 19:07:54.391413 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.458950 (+ 67537us) service_pool.cc:227] Handling call
0812 19:07:54.498385 (+ 39435us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:54.498741 (+ 356us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:54.498770 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:54.499201 (+ 431us) log.cc:842] Serialized 26 byte log entry
0812 19:07:56.024430 (+1525229us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:56.024471 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.024540 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:57.725741 (+1701201us) raft_consensus.cc:1671] finished
0812 19:07:57.725778 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.726857 (+ 1079us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":8796,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":14663,"thread_start_us":1524932,"threads_started":1,"wal-append.queue_time_us":2231473}
I20260812 19:07:57.729404 1852 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.338054 1904 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:57.725778 29445 rpcz_store.cc:276] 0812 19:07:53.479711 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.494764 (+ 15053us) service_pool.cc:227] Handling call
0812 19:07:53.495097 (+ 333us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:54.205212 (+710115us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:54.205248 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:54.205785 (+ 537us) log.cc:842] Serialized 26 byte log entry
0812 19:07:56.143754 (+1937969us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:56.143793 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:56.143860 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:57.724719 (+1580859us) raft_consensus.cc:1671] finished
0812 19:07:57.724764 (+ 45us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.725004 (+ 240us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":213902,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":2531452,"threads_started":2,"wal-append.queue_time_us":2649861}
I20260812 19:07:57.731491 1904 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:07:57.244305 1292 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:57.613129 1826 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:57.215759 1458 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:57.734092 29457 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.280s user 0.006s sys 0.025s
I20260812 19:07:57.720667 1829 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:57.734577 29457 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.327s user 0.006s sys 0.025s
I20260812 19:07:57.198716 1160 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:57.615510 29579 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.436s user 0.008s sys 0.011s
I20260812 19:07:57.251788 1478 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.642127 1878 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.838s user 0.004s sys 0.028s
W20260812 19:07:57.740677 1878 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.937s user 0.004s sys 0.028s
W20260812 19:07:57.642581 1649 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.728s user 0.012s sys 0.013s
I20260812 19:07:56.890164 1622 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.740901 1458 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.742666 1160 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.742193 1945 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:57.743321 1622 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:57.635946 30685 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.815s user 0.011s sys 0.006s
I20260812 19:07:57.645762 29451 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
W20260812 19:07:57.638805 29575 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.231s user 0.000s sys 0.013s
W20260812 19:07:57.637982 29580 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.225s user 0.002s sys 0.011s
I20260812 19:07:57.646375 1908 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:57.655237 1934 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:57.635512 29282 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.230s user 0.018s sys 0.013s
W20260812 19:07:57.749785 29282 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.344s user 0.018s sys 0.013s
I20260812 19:07:57.736995 1826 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.664613 1455 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.665899 1496 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.005s sys 0.007s
W20260812 19:07:57.666280 1470 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.968s user 0.007s sys 0.023s
W20260812 19:07:57.666603 1362 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 2.079s user 0.005s sys 0.010s
I20260812 19:07:57.668511 1448 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.669394 1502 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.275s user 0.002s sys 0.015s
I20260812 19:07:57.733218 1904 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.676806 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.695778 1722 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:57.678500 1726 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:57.678306 1736 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.696887 1772 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.705683 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:57.700744 1897 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.368s user 0.000s sys 0.015s
I20260812 19:07:57.694509 1889 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:57.676121 1717 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:57.711531 1507 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.701761 1770 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.697029 998 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:57.755789 1922 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:57.775518 1922 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.776705 1717 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:57.366813 1870 log.cc:924] Time spent T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.172s user 0.000s sys 0.001s
I20260812 19:07:57.217787 1885 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260812 19:07:57.743884 1945 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.728971 29577 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:57.781921 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 715) took 1521 ms (client timeout 1621 ms). Trace:
W20260812 19:07:57.739511 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 46178.7us
I20260812 19:07:57.735011 1292 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.782282 29456 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 5. Ops: []
W20260812 19:07:57.782318 29577 rpcz_store.cc:269] 0812 19:07:56.260033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.724581 (+1464548us) service_pool.cc:227] Handling call
0812 19:07:57.781843 (+ 57262us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:57.784142 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:57.738934 29335 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:57.746779 1622 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:57.747104 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2206) took 3281 ms. Trace:
I20260812 19:07:57.748582 1908 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:57.750759 1649 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.870s user 0.012s sys 0.013s
W20260812 19:07:57.669776 1638 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.436s user 0.019s sys 0.034s
I20260812 19:07:57.756197 1025 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:57.749933 1934 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:57.754760 1362 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 2.180s user 0.005s sys 0.011s
W20260812 19:07:57.755627 1502 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s user 0.002s sys 0.016s
I20260812 19:07:57.795030 1736 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.787861 29575 rpcz_store.cc:276] 0812 19:07:54.465274 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.359915 (+894641us) service_pool.cc:227] Handling call
0812 19:07:55.360257 (+ 342us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:55.360478 (+ 221us) raft_consensus.cc:1520] Early marking committed up to index 5
0812 19:07:56.407580 (+1047102us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.746434 (+1338854us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.746557 (+ 123us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:57.746712 (+ 155us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.746841 (+ 129us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.747036 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1906915,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":577,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":230,"thread_start_us":2332427,"threads_started":2,"wal-append.queue_time_us":960885}
I20260812 19:07:57.729784 1899 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.750958 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 856) took 2274 ms. Trace:
W20260812 19:07:57.754182 1496 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.627s user 0.005s sys 0.007s
W20260812 19:07:57.754446 1470 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s user 0.007s sys 0.023s
I20260812 19:07:57.796314 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:57.786224 30629 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:57.758026 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2096) took 1348 ms. Trace:
I20260812 19:07:57.758635 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2189) took 4207 ms. Trace:
I20260812 19:07:57.754637 1826 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:07:57.771325 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.257s user 0.005s sys 0.011s
W20260812 19:07:57.773814 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 146996us
I20260812 19:07:57.774461 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2204) took 3315 ms. Trace:
I20260812 19:07:57.778680 1726 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:57.779390 1885 raft_consensus.cc:3032] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:07:57.244165 1895 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:57.826927 1895 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.796207 30679 rpcz_store.cc:276] 0812 19:07:55.476287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.572558 (+ 96271us) service_pool.cc:227] Handling call
0812 19:07:55.706263 (+133705us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:55.781037 (+ 74774us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:55.812325 (+ 31288us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:55.820353 (+ 8028us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:55.824726 (+ 4373us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.750368 (+1925642us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54005483d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:57.750444 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.750736 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":122075520}
I20260812 19:07:57.780897 1932 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.743949 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.508s user 0.007s sys 0.012s
W20260812 19:07:57.832461 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.653s user 0.008s sys 0.012s
I20260812 19:07:57.809327 29572 rpcz_store.cc:276] 0812 19:07:53.551029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.555917 (+ 4888us) service_pool.cc:227] Handling call
0812 19:07:53.586169 (+ 30252us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:53.772038 (+185869us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:53.772078 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:54.118125 (+346047us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b2c00032a08. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c687fde11 00007f9c6880d442 00007f9c69e49c11 00007f9c69e0e58a 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772
0812 19:07:54.138390 (+ 20265us) log.cc:842] Serialized 26 byte log entry
0812 19:07:55.915482 (+1777092us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:55.915523 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:55.915607 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:57.758028 (+1842421us) raft_consensus.cc:1671] finished
0812 19:07:57.758068 (+ 40us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.758433 (+ 365us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":39463,"raft.queue_time_us":56003,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":33483,"spinlock_wait_cycles":15639424,"thread_start_us":1921434,"threads_started":2,"wal-append.queue_time_us":2269959}
I20260812 19:07:57.835122 1726 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:57.780656 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.849s user 0.010s sys 0.007s
I20260812 19:07:57.835515 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:57.835918 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.997s user 0.011s sys 0.007s
W20260812 19:07:57.836357 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 997280us
I20260812 19:07:57.837308 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 860) took 1020 ms. Trace:
W20260812 19:07:57.786000 29281 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.752s user 0.004s sys 0.015s
I20260812 19:07:57.839475 1539 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.784727 1889 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.840046 1829 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.839502 29281 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.908s user 0.004s sys 0.015s
W20260812 19:07:57.840794 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 1.9095e+06us
I20260812 19:07:57.784377 1904 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:57.837721 30685 rpcz_store.cc:276] 0812 19:07:56.817049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.817162 (+ 113us) service_pool.cc:227] Handling call
0812 19:07:56.820612 (+ 3450us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:56.820854 (+ 242us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:57.744294 (+923440us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.836750 (+ 92456us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.836837 (+ 87us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:57.836931 (+ 94us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.837035 (+ 104us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.837221 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":546803,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":18795,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":988691,"threads_started":2}
I20260812 19:07:57.844697 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.816468 1622 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.846903 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.793606 1025 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:57.848024 1622 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.848680 1292 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.849071 1025 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:57.849519 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:57.850129 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:07:57.797204 29575 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 723) took 1407 ms (client timeout 1628 ms). Trace:
W20260812 19:07:57.815814 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.408s user 0.005s sys 0.011s
W20260812 19:07:57.816505 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 724) took 1422 ms (client timeout 1550 ms). Trace:
I20260812 19:07:57.787096 1922 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:57.798494 1638 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.598s user 0.020s sys 0.034s
I20260812 19:07:57.809142 29446 rpcz_store.cc:276] 0812 19:07:56.409512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.416028 (+ 6516us) service_pool.cc:227] Handling call
0812 19:07:57.753134 (+1337106us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:07:57.753473 (+ 339us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:57.753939 (+ 466us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:57.757441 (+ 3502us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.757514 (+ 73us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:57.757582 (+ 68us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.757658 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.757794 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":15260928}
I20260812 19:07:57.819043 30675 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 1. Current term is 2. Ops: []
I20260812 19:07:57.830916 1820 raft_consensus.cc:3032] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:07:57.846954 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:57.850432 30674 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:57.784608 29402 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.983s user 0.010s sys 0.007s
W20260812 19:07:57.786449 29459 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.266s user 0.015s sys 0.033s
I20260812 19:07:57.815896 1885 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:07:57.851089 29577 rpcz_store.cc:269] 0812 19:07:56.394343 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.783726 (+1389383us) service_pool.cc:227] Handling call
0812 19:07:57.816437 (+ 32711us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:57.895131 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.50169e+06us
I20260812 19:07:57.881704 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39326 (request call id 2089) took 1480 ms. Trace:
I20260812 19:07:57.882189 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:57.980887 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 50547.9us
I20260812 19:07:57.981379 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.810652 29578 rpcz_store.cc:276] 0812 19:07:54.458832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.745971 (+287139us) service_pool.cc:227] Handling call
0812 19:07:54.746581 (+ 610us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:54.752320 (+ 5739us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:54.752351 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:54.753648 (+ 1297us) log.cc:842] Serialized 590 byte log entry
0812 19:07:57.183838 (+2430190us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:57.183889 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.183957 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:57.699330 (+515373us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:07:57.773760 (+ 74430us) raft_consensus.cc:1671] finished
0812 19:07:57.773785 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.774229 (+ 444us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4820,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"spinlock_wait_cycles":61289856,"thread_start_us":2429684,"threads_started":1,"wal-append.queue_time_us":2494213}
I20260812 19:07:57.883387 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:57.797457 1899 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:57.980003 29451 rpcz_store.cc:276] 0812 19:07:56.401334 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.403720 (+ 2386us) service_pool.cc:227] Handling call
0812 19:07:57.881407 (+1477687us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000fa0d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:57.881598 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2548812032}
W20260812 19:07:57.851114 29575 rpcz_store.cc:269] 0812 19:07:56.390706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.795917 (+1405211us) service_pool.cc:227] Handling call
0812 19:07:57.797129 (+ 1212us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:57.985016 29459 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.568s user 0.016s sys 0.038s
W20260812 19:07:57.985209 29402 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.282s user 0.012s sys 0.011s
I20260812 19:07:57.927330 29338 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:07:57.980304 30685 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 2.
I20260812 19:07:57.978343 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:57.836192 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2209) took 3204 ms. Trace:
W20260812 19:07:57.932597 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 146663us
I20260812 19:07:57.990242 1722 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:57.990849 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 858) took 2449 ms. Trace:
I20260812 19:07:57.991348 30671 rpcz_store.cc:276] 0812 19:07:55.477298 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.562382 (+ 85084us) service_pool.cc:227] Handling call
0812 19:07:55.562697 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:55.825032 (+262335us) raft_consensus.cc:1520] Early marking committed up to index 4
0812 19:07:55.825072 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:55.825113 (+ 41us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:55.825151 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.926401 (+2101250us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54005a0854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:57.926466 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.926716 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":602688384}
I20260812 19:07:57.992944 1722 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:57.993904 30684 raft_consensus.cc:2354] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:57.990686 1908 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:57.836290 29572 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:57.799285 1934 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:57.996285 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.999843 30681 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:57.836166 1810 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:58.001093 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 728) took 1261 ms. Trace:
I20260812 19:07:58.001112 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 914) took 1174 ms. Trace:
I20260812 19:07:58.001577 29572 rpcz_store.cc:276] 0812 19:07:56.737442 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.835127 (+1097685us) service_pool.cc:227] Handling call
0812 19:07:57.998435 (+163308us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:57.943516 1502 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:57.979526 1820 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:58.001643 30670 rpcz_store.cc:276] 0812 19:07:56.824533 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.838874 (+ 14341us) service_pool.cc:227] Handling call
0812 19:07:57.846629 (+1007755us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:57.999069 (+152440us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11318528}
W20260812 19:07:57.953234 29574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2162) took 3630 ms (client timeout 1725 ms). Trace:
I20260812 19:07:57.898144 1772 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:57.882902 1945 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:57.870162 1895 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:07:58.008327 29574 rpcz_store.cc:269] 0812 19:07:54.323166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.404630 (+ 81464us) service_pool.cc:227] Handling call
0812 19:07:57.952837 (+3548207us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400136354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:57.953035 (+ 198us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2579922688}
I20260812 19:07:58.008728 29341 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:57.987617 29579 rpcz_store.cc:276] 0812 19:07:54.631659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.151076 (+1519417us) service_pool.cc:227] Handling call
0812 19:07:56.151401 (+ 325us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:56.151618 (+ 217us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:57.739635 (+1588017us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.835659 (+ 96024us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.835748 (+ 89us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:57.835855 (+ 107us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.835961 (+ 106us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:57.836126 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1176199,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":20023,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":149,"thread_start_us":1611120,"threads_started":2,"wal-append.queue_time_us":159108}
I20260812 19:07:58.014861 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.015568 29579 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:07:58.016707 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.007966 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:58.019028 1806 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:58.019198 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2212) took 2770 ms. Trace:
I20260812 19:07:58.019604 29579 rpcz_store.cc:276] 0812 19:07:55.248426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.017771 (+2769345us) service_pool.cc:227] Handling call
0812 19:07:58.018234 (+ 463us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:58.018418 (+ 184us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:58.018440 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:58.018501 (+ 61us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:07:58.018534 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.018582 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.019125 (+ 543us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:58.020620 29281 connection.cc:232] RPC call timeout handler was delayed by -1.7161644869484007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260812 19:07:58.021545 1717 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:58.005079 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:58.021517 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 714) took 1762 ms. Trace:
I20260812 19:07:58.019083 29577 raft_consensus.cc:3037] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:58.026332 30679 raft_consensus.cc:2354] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:58.003866 1502 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:58.027896 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.028450 29331 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:58.013525 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.030985 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2098) took 1621 ms. Trace:
I20260812 19:07:58.027710 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:58.022486 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:58.038331 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 89801.7us
I20260812 19:07:58.038581 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.038759 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 912) took 1214 ms. Trace:
I20260812 19:07:58.039158 30676 rpcz_store.cc:276] 0812 19:07:56.824055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.837788 (+ 13733us) service_pool.cc:227] Handling call
0812 19:07:58.022172 (+1184384us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:58.038692 (+ 16520us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13966592}
W20260812 19:07:58.039963 1896 log.cc:924] Time spent T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.684s user 0.000s sys 0.013s
I20260812 19:07:58.022799 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:57.895650 1886 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:58.045220 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.032618 1502 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:07:58.053297 29281 connection.cc:232] RPC call timeout handler was delayed by -1.0208955781999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2437s for a response
I20260812 19:07:58.031425 29459 rpcz_store.cc:276] 0812 19:07:56.409999 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.416278 (+ 6279us) service_pool.cc:227] Handling call
0812 19:07:56.416551 (+ 273us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:58.023644 (+1607093us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:58.030546 (+ 6902us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:58.030605 (+ 59us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:58.030653 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.030753 (+ 100us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.030911 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":898085,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":142,"thread_start_us":1570257,"threads_started":1}
I20260812 19:07:58.071673 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2429) took 3438 ms. Trace:
I20260812 19:07:58.072181 29456 rpcz_store.cc:276] 0812 19:07:54.632773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.644640 (+ 11867us) service_pool.cc:227] Handling call
0812 19:07:54.645025 (+ 385us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:57.810313 (+3165288us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.071315 (+261002us) spinlock_profiling.cc:243] Waited 3.12 s on lock 0x7b54001ab154. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:58.071587 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7181411072}
I20260812 19:07:58.076678 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.022481 29579 rpcz_store.cc:276] 0812 19:07:56.259402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.020507 (+1761105us) service_pool.cc:227] Handling call
0812 19:07:58.020880 (+ 373us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:58.021201 (+ 321us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:58.021223 (+ 22us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:58.021277 (+ 54us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:07:58.021303 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.021347 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.021448 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:58.140654 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.122947 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.107817 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.140218 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.111214 30686 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:58.125136 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.187340 29577 raft_consensus.cc:2454] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
W20260812 19:07:58.109520 29569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2154) took 3829 ms (client timeout 2279 ms). Trace:
I20260812 19:07:58.175226 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.106607 29567 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:58.117022 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:58.125895 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2211) took 2878 ms. Trace:
I20260812 19:07:58.109723 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2093) took 1707 ms. Trace:
I20260812 19:07:58.102147 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2210) took 3470 ms. Trace:
I20260812 19:07:58.109213 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.130627 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.106389 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.183938 30681 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:58.081700 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34908 (request call id 31) took 1321 ms. Trace:
I20260812 19:07:58.124068 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.142776 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.176712 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.103737 29578 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:07:58.176298 30629 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:58.141983 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2092) took 1739 ms. Trace:
I20260812 19:07:58.176014 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.126212 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 906) took 3055 ms. Trace:
I20260812 19:07:58.126597 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.110493 29281 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:58.255421 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 729) took 1441 ms. Trace:
I20260812 19:07:58.102936 29338 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.256227 29578 rpcz_store.cc:276] 0812 19:07:56.814118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.000999 (+1186881us) service_pool.cc:227] Handling call
0812 19:07:58.255360 (+254361us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:58.234355 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 726) took 1824 ms (client timeout 2064 ms). Trace:
W20260812 19:07:58.129266 1949 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
W20260812 19:07:58.258391 29577 rpcz_store.cc:269] 0812 19:07:56.410719 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.995626 (+1584907us) service_pool.cc:227] Handling call
0812 19:07:58.234293 (+238667us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:58.256704 30675 rpcz_store.cc:276] 0812 19:07:55.071039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:55.076121 (+ 5082us) service_pool.cc:227] Handling call
0812 19:07:55.169687 (+ 93566us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:57.998883 (+2829196us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.120049 (+121166us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54007526d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:58.126151 (+ 6102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6069884800}
I20260812 19:07:58.256300 29576 rpcz_store.cc:276] 0812 19:07:55.247777 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.257068 (+1009291us) service_pool.cc:227] Handling call
0812 19:07:56.257413 (+ 345us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:57.175528 (+918115us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:57.175557 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.175595 (+ 38us) raft_consensus.cc:1631] Marking committed up to 2
0812 19:07:57.175631 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.125473 (+949842us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000df554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:58.125533 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.125713 (+ 180us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1587046656}
I20260812 19:07:58.259608 30671 raft_consensus.cc:2354] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:58.238363 30670 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:58.254789 29458 rpcz_store.cc:276] 0812 19:07:56.402404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.406323 (+ 3919us) service_pool.cc:227] Handling call
0812 19:07:56.406852 (+ 529us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:57.721338 (+1314486us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.109316 (+387978us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000c5dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:58.109654 (+ 338us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3013573504}
I20260812 19:07:58.255362 30684 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:07:58.234326 29569 rpcz_store.cc:269] 0812 19:07:54.279854 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:54.279975 (+ 121us) service_pool.cc:227] Handling call
0812 19:07:54.627325 (+347350us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b5800004298. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:07:58.109132 (+3481807us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b54000df054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:58.109302 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5025955072}
W20260812 19:07:58.108634 1954 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.197s user 0.000s sys 0.002s
I20260812 19:07:58.268901 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.102432 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.271112 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.110154 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.124682 1934 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:58.082827 1741 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:58.273455 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.280349 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.261323 29450 rpcz_store.cc:276] 0812 19:07:56.760136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.831958 (+ 71822us) service_pool.cc:227] Handling call
0812 19:07:58.081343 (+1249385us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54000fa5d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e08bc3 00007f9c6ed5281a 00007f9c68451d04 00007f9c68451c92 00007f9c68451bc1 00007f9c68451b3c 00007f9c68450df2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:58.081544 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1581694848}
I20260812 19:07:58.270128 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:58.255128 29457 rpcz_store.cc:276] 0812 19:07:56.402145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.406183 (+ 4038us) service_pool.cc:227] Handling call
0812 19:07:56.407645 (+ 1462us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:07:57.735297 (+1327652us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:57.735351 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:07:57.778342 (+ 42991us) log.cc:842] Serialized 52 byte log entry
0812 19:07:57.821967 (+ 43625us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:57.822007 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:57.832159 (+ 10152us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:58.117160 (+285001us) raft_consensus.cc:1671] finished
0812 19:07:58.117180 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.141896 (+ 24716us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":814578,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":11284,"thread_start_us":1362438,"threads_started":2,"wal-append.queue_time_us":215644}
I20260812 19:07:58.274657 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.303406 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:58.275676 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 99489us
I20260812 19:07:58.276513 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.109724 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.102262 1886 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:58.280048 29572 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:07:58.255926 29580 rpcz_store.cc:276] 0812 19:07:54.631973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:56.180364 (+1548391us) service_pool.cc:227] Handling call
0812 19:07:56.228401 (+ 48037us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:56.348470 (+120069us) raft_consensus.cc:1520] Early marking committed up to index 4
0812 19:07:57.747593 (+1399123us) log.cc:842] Serialized 14 byte log entry
0812 19:07:57.811270 (+ 63677us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:57.821416 (+ 10146us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:57.826464 (+ 5048us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.101875 (+275411us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b5400135954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:07:58.101919 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.102057 (+ 138us) inbound_call.cc:173] Queueing success response
0812 19:07:58.108135 (+ 6078us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b24001e5c18. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66d4acf4 00007f9c66d48dfb 00007f9c66dd7559 00007f9c66dd71e5 00007f9c66dd682d 00007f9c69a7946e 00007f9c69a90c0c 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551
Metrics: {"mutex_wait_us":14360,"prepare.queue_time_us":993052,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":1241,"raft.queue_time_us":18505,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6311,"spinlock_wait_cycles":233204352,"thread_start_us":1354762,"threads_started":2,"wal-append.queue_time_us":150476}
I20260812 19:07:58.297189 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.275102 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:58.279758 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:58.268267 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 731) took 1271 ms. Trace:
I20260812 19:07:58.308892 29332 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:58.309525 1934 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:58.309872 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.310798 29332 raft_consensus.cc:2379] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:58.311596 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.312321 29332 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:07:58.313104 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.309777 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:58.314121 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 203732us
I20260812 19:07:58.306476 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.082681 1807 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:58.082468 1772 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:58.130307 1820 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:58.143462 1885 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:58.357312 29579 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:07:58.357226 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.324440 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.314224 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:58.410562 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 86380.5us
W20260812 19:07:58.363168 29567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 716) took 2065 ms (client timeout 1555 ms). Trace:
I20260812 19:07:58.308687 29575 rpcz_store.cc:276] 0812 19:07:56.997212 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.124773 (+1127561us) service_pool.cc:227] Handling call
0812 19:07:58.268173 (+143400us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:58.923107 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.926944 29575 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:58.667587 29176 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:58.314430 1741 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:58.930534 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.930387 29176 catalog_manager.cc:5495] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:07:58.606277 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 172193us
I20260812 19:07:58.931720 1741 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:58.931283 29575 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
W20260812 19:07:58.923712 29567 rpcz_store.cc:269] 0812 19:07:56.260581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:57.705039 (+1444458us) service_pool.cc:227] Handling call
0812 19:07:58.325964 (+620925us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54003e0154. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:58.326133 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":722932480}
I20260812 19:07:58.938514 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.938711 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.939774 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:58.939864 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.916343 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.327692 30684 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:58.412736 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.314224 1649 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:58.404942 29577 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:58.451892 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.818671 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:58.591627 1810 consensus_meta.cc:287] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.587s user 0.010s sys 0.001s
I20260812 19:07:58.417819 29578 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:58.608278 1839 consensus_queue.cc:1046] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.475s
I20260812 19:07:58.323814 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.927572 1885 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:58.359081 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.936084 1772 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:07:58.690024 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 45664.5us
I20260812 19:07:58.941365 29567 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:07:58.933528 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:58.439430 1958 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.097s user 0.001s sys 0.000s
I20260812 19:07:58.952153 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.124477 1858 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:58.307128 30674 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.946273 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.302824 1945 raft_consensus.cc:2781] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:58.957476 29580 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:07:58.961594 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:58.962441 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:58.965119 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.965914 29326 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:58.952425 29574 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.944164 1649 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:58.943933 1895 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.018s sys 0.035s
I20260812 19:07:58.968765 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.970638 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.793488 1722 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:58.971526 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.955782 1934 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:58.975581 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:58.977766 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:58.978880 29335 raft_consensus.cc:2354] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9.
W20260812 19:07:58.971901 1895 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.554s user 0.019s sys 0.036s
I20260812 19:07:58.980268 29576 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.981581 29332 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:58.982666 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.984990 29335 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:58.972191 1649 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:58.980033 1945 raft_consensus.cc:686] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:58.988408 29574 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:07:58.989244 29579 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.986200 29577 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:58.993777 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.995390 29326 raft_consensus.cc:2379] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:58.995797 29335 raft_consensus.cc:2379] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:58.995873 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 722) took 2691 ms. Trace:
I20260812 19:07:58.997444 29574 rpcz_store.cc:276] 0812 19:07:56.297942 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.972777 (+2674835us) service_pool.cc:227] Handling call
0812 19:07:58.979989 (+ 7212us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:58.988802 (+ 8813us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.988979 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:58.995465 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:58.996217 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.000798 29576 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:59.000524 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 719) took 2732 ms. Trace:
I20260812 19:07:59.003347 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.003001 29572 rpcz_store.cc:276] 0812 19:07:56.262608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.948653 (+2686045us) service_pool.cc:227] Handling call
0812 19:07:58.991890 (+ 43237us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:58.994923 (+ 3033us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:58.994954 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:58.994995 (+ 41us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:58.995031 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:58.995083 (+ 52us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:58.995191 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.004408 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.003772 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.009254 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:59.011433 29335 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:07:59.014369 29338 raft_consensus.cc:2379] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:59.012609 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.018829 29326 raft_consensus.cc:2354] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:07:59.019455 29575 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260812 19:07:59.020259 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 718) took 2758 ms. Trace:
I20260812 19:07:59.019598 29339 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 2.
I20260812 19:07:59.022056 29575 rpcz_store.cc:276] 0812 19:07:56.261810 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.958588 (+2696778us) service_pool.cc:227] Handling call
0812 19:07:58.986166 (+ 27578us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:59.019957 (+ 33791us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.020202 (+ 245us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:58.989020 1722 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.025733 29580 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:07:59.026393 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 721) took 2736 ms. Trace:
I20260812 19:07:59.028833 29580 rpcz_store.cc:276] 0812 19:07:56.289779 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.962279 (+2672500us) service_pool.cc:227] Handling call
0812 19:07:58.991193 (+ 28914us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.026154 (+ 34961us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.026341 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.032737 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 720) took 2742 ms. Trace:
I20260812 19:07:59.034116 29569 rpcz_store.cc:276] 0812 19:07:56.286436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.955425 (+2668989us) service_pool.cc:227] Handling call
0812 19:07:58.995231 (+ 39806us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.024205 (+ 28974us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.024234 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.024301 (+ 67us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:59.028618 (+ 4317us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.028678 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.028786 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":275712}
I20260812 19:07:59.032653 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:58.994877 1945 consensus_queue.cc:237] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.042446 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.045137 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2213) took 1151 ms. Trace:
I20260812 19:07:59.054536 29572 rpcz_store.cc:276] 0812 19:07:57.894010 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.004225 (+1110215us) service_pool.cc:227] Handling call
0812 19:07:59.011748 (+ 7523us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.038595 (+ 26847us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.038614 (+ 19us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.038642 (+ 28us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:59.044942 (+ 6300us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.045004 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.045092 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.057509 29574 raft_consensus.cc:1226] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:59.060386 1722 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:59.060812 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 717) took 2799 ms. Trace:
W20260812 19:07:59.063707 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 132236us
I20260812 19:07:59.063867 29578 rpcz_store.cc:276] 0812 19:07:56.261254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.952358 (+2691104us) service_pool.cc:227] Handling call
0812 19:07:58.990929 (+ 38571us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.001195 (+ 10266us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.001221 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.005086 (+ 3865us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:59.005129 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.005183 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.060610 (+ 55427us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b54002003d0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:59.060744 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":35927808}
I20260812 19:07:59.065852 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 730) took 2185 ms. Trace:
I20260812 19:07:59.066363 29574 rpcz_store.cc:276] 0812 19:07:56.880189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.003481 (+2123292us) service_pool.cc:227] Handling call
0812 19:07:59.025484 (+ 22003us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.065401 (+ 39917us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.065789 (+ 388us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.066325 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2214) took 1171 ms. Trace:
I20260812 19:07:59.068190 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.069123 29576 rpcz_store.cc:276] 0812 19:07:57.894563 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.006793 (+1112230us) service_pool.cc:227] Handling call
0812 19:07:59.019000 (+ 12207us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.066007 (+ 47007us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.066046 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.066092 (+ 46us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:59.066129 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.066184 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.066276 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.071399 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 732) took 1026 ms. Trace:
I20260812 19:07:59.072681 29569 rpcz_store.cc:276] 0812 19:07:58.047351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.038918 (+991567us) service_pool.cc:227] Handling call
0812 19:07:59.064592 (+ 25674us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.068927 (+ 4335us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.068945 (+ 18us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.071141 (+ 2196us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:59.071177 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.071236 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.071342 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.069502 30623 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.077098 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 725) took 2671 ms. Trace:
I20260812 19:07:59.079587 29579 rpcz_store.cc:276] 0812 19:07:56.406333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.996441 (+2590108us) service_pool.cc:227] Handling call
0812 19:07:59.020837 (+ 24396us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.071601 (+ 50764us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.071630 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.076858 (+ 5228us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:59.076904 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.076965 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.077058 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3053}
I20260812 19:07:59.083238 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2226) took 1012 ms. Trace:
I20260812 19:07:59.083636 29572 rpcz_store.cc:276] 0812 19:07:58.071194 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.057841 (+986647us) service_pool.cc:227] Handling call
0812 19:07:59.071706 (+ 13865us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.082981 (+ 11275us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.082998 (+ 17us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.083036 (+ 38us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:59.083057 (+ 21us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.083095 (+ 38us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.083193 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.077872 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2216) took 1182 ms. Trace:
I20260812 19:07:59.085884 29580 rpcz_store.cc:276] 0812 19:07:57.895336 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.030123 (+1134787us) service_pool.cc:227] Handling call
0812 19:07:59.065135 (+ 35012us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.072091 (+ 6956us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.072117 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.077694 (+ 5577us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:07:59.077724 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.077765 (+ 41us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.077832 (+ 67us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:07:59.090801 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 649883us
I20260812 19:07:59.094166 30629 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:59.094487 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.076782 1807 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.102955 30629 leader_election.cc:398] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9.
I20260812 19:07:59.101785 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.103360 30629 leader_election.cc:401] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260812 19:07:59.085438 1722 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.103470 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.108479 29574 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:07:59.107362 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.110666 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.122875 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.121282 1515 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:59.124958 1515 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:59.125316 1515 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:59.123453 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.133759 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.137301 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.142067 29457 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd"
dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
mode: GRACEFUL
new_leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
from {username='slave'} at 127.0.0.1:34908
I20260812 19:07:59.143150 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.145471 29281 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.116901 1722 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:59.146325 29530 leader_election.cc:334] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.699s (SENT)
I20260812 19:07:59.135422 1904 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:59.141441 1502 consensus_meta.cc:287] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.017s user 0.012s sys 0.000s
I20260812 19:07:59.145918 29457 raft_consensus.cc:595] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Received request to transfer leadership to TS dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.150434 29457 raft_consensus.cc:603] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:07:59.153199 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.144232 1638 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.155285 30684 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:59.159898 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.155558 1966 log.cc:924] Time spent T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
I20260812 19:07:59.158629 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260812 19:07:59.164052 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.162953 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.164188 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.155885 1638 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:59.166661 30684 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:07:59.167644 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.150908 1904 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.169754 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.170944 30686 raft_consensus.cc:2396] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:07:59.164446 1478 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:59.152501 1502 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.164573 1362 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:07:59.167171 1638 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.176193 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:59.176796 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.175645 1362 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:07:59.179131 1960 log.cc:924] Time spent T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.097s user 0.000s sys 0.001s
I20260812 19:07:59.181244 1362 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:59.181664 1967 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.182278 1967 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:59.182832 1895 raft_consensus.cc:3032] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:59.183319 1895 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:59.184796 1967 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.173707 1478 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:07:59.183168 29528 leader_election.cc:334] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.591s (SENT)
I20260812 19:07:59.185572 1478 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.185863 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.185106 30671 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:59.180742 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.170360 1839 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:59.211793 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.221196 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 114272us
I20260812 19:07:59.211191 1839 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.222816 1839 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:59.223671 1904 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.227742 1502 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.229751 1973 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9.
I20260812 19:07:59.180824 1820 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:59.184268 1895 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.231201 1895 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:07:59.232363 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 132427us
I20260812 19:07:59.242724 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.243300 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.173444 1971 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:59.246402 29576 raft_consensus.cc:2379] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:59.211140 1741 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:59.250672 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.259073 30629 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:07:59.249409 29399 connection.cc:232] RPC call timeout handler was delayed by -2.2796509340405464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260812 19:07:59.259796 29399 connection.cc:232] RPC call timeout handler was delayed by -2.00605093408376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260812 19:07:59.260375 29399 connection.cc:232] RPC call timeout handler was delayed by -1.7810509339906275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2279s for a response
W20260812 19:07:59.260676 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1771509340032935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29500000000000004s for a response
I20260812 19:07:59.257359 29574 raft_consensus.cc:2379] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:59.249162 30623 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.256246 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.250933 29403 leader_election.cc:334] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.522s (SENT)
I20260812 19:07:59.268723 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.269410 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:59.270032 30670 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.261723 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.256321 1971 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:59.270797 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.271376 30681 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:59.272401 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.272971 29456 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:07:59.273164 29451 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:59.274068 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.274397 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.274605 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.275043 30675 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:07:59.275954 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.270291 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.275038 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.277557 29569 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:59.275822 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.278438 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.272971 29457 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:07:59.232559 1967 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:59.282296 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 124231us
I20260812 19:07:59.282886 30671 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 8: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:07:59.249493 1904 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:59.278451 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:07:59.284328 29446 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
W20260812 19:07:59.284794 29403 leader_election.cc:334] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.691s (SENT)
W20260812 19:07:59.289726 29403 leader_election.cc:334] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.698s (SENT)
I20260812 19:07:59.293076 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:07:59.295027 1983 log.cc:924] Time spent T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.078s user 0.000s sys 0.005s
I20260812 19:07:59.295233 29572 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:59.296365 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.296923 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.298151 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.298241 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.297494 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.299244 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.298931 29574 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:07:59.270756 1971 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.299381 29572 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:07:59.304180 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.303895 29403 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.295038 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.299958 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.305593 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.310585 29450 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.306635 29332 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:07:59.317524 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.318205 29579 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.316886 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.320701 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.321370 29574 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:59.316900 29576 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:07:59.319976 29456 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9.
I20260812 19:07:59.326604 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.325502 29445 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:59.335002 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.305789 1978 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:59.334849 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:07:59.342474 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.347421 29459 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.349582 30670 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
W20260812 19:07:59.349851 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 54617us
I20260812 19:07:59.358408 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.350770 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.359751 29450 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.357849 29446 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:59.360811 29445 raft_consensus.cc:2379] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:07:59.361737 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 74189.6us
I20260812 19:07:59.335428 1722 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:07:59.337169 1717 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:59.368106 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 802) took 1055 ms. Trace:
I20260812 19:07:59.337721 1978 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:59.320415 1967 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:59.366308 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 733) took 1269 ms. Trace:
I20260812 19:07:59.376259 29578 rpcz_store.cc:276] 0812 19:07:58.121115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.069625 (+948510us) service_pool.cc:227] Handling call
0812 19:07:59.078541 (+ 8916us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.100670 (+ 22129us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:07:59.255803 (+155133us) log.cc:842] Serialized 14 byte log entry
0812 19:07:59.360158 (+104355us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.360207 (+ 49us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:07:59.360251 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.360310 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.360411 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3112,"prepare.queue_time_us":144820,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":13299,"thread_start_us":180057,"threads_started":2}
W20260812 19:07:59.377400 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 128348us
I20260812 19:07:59.378181 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.382788 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.383498 29459 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:07:59.384797 30629 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:59.386897 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:07:59.386107 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 96407.8us
I20260812 19:07:59.393698 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.397197 29403 leader_election.cc:334] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.735s (SENT)
I20260812 19:07:59.397657 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.367254 1807 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:59.398226 29459 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:07:59.368536 29326 rpcz_store.cc:276] 0812 19:07:58.312923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.021515 (+708592us) service_pool.cc:227] Handling call
0812 19:07:59.031801 (+ 10286us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.032007 (+ 206us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:59.064173 (+ 32166us) log.cc:842] Serialized 14 byte log entry
0812 19:07:59.341769 (+277596us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b18003ccd00. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ed4345 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66eef155 00007f9c69e063d0 00007f9c69e18576 00007f9c69e290d7 00007f9c69e29058
0812 19:07:59.367786 (+ 26017us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.367832 (+ 46us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:59.367881 (+ 49us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.367939 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.368052 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":28316,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2074,"raft.queue_time_us":42054,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"spinlock_wait_cycles":70163584,"thread_start_us":162775,"threads_started":2,"wal-append.queue_time_us":147177}
I20260812 19:07:59.339656 1904 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.406951 1968 raft_consensus.cc:2741] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260812 19:07:59.407790 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.408299 1979 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:07:59.375129 1978 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:59.408591 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.408811 1807 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.410277 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.411021 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.415848 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.416859 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.417095 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.417912 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.418606 29332 raft_consensus.cc:2379] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:59.418694 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:59.419289 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.419790 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.420346 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.411791 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.411597 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.412375 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.375032 1984 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:59.404716 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 727) took 2768 ms. Trace:
I20260812 19:07:59.473140 29579 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:59.415104 29341 raft_consensus.cc:2379] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.413096 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.420538 29335 raft_consensus.cc:2354] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:07:59.412549 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.478622 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 59753.4us
I20260812 19:07:59.421536 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.348640 1515 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.444375 29332 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:59.425730 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:59.470635 29567 consensus_meta.cc:287] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.503s user 0.009s sys 0.006s
I20260812 19:07:59.467804 29457 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.468253 29450 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.476485 29445 raft_consensus.cc:2354] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5.
I20260812 19:07:59.467967 29452 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:59.479660 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:07:59.483441 29403 leader_election.cc:334] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.831s (SENT)
I20260812 19:07:59.484804 29403 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.485663 1717 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260812 19:07:59.486150 29403 leader_election.cc:334] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.893s (SENT)
I20260812 19:07:59.483217 29567 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:59.487131 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2221) took 1434 ms. Trace:
I20260812 19:07:59.487421 29567 rpcz_store.cc:276] 0812 19:07:58.053176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.939494 (+886318us) service_pool.cc:227] Handling call
0812 19:07:59.487076 (+547582us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:07:59.474449 30623 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.487807 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:59.489007 29174 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55520 (request call id 375) took 12336 ms (client timeout 15000 ms). Trace:
I20260812 19:07:59.491238 1839 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:59.489440 29174 rpcz_store.cc:269] 0812 19:07:47.132245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.141822 (+ 9577us) service_pool.cc:227] Handling call
0812 19:07:59.430202 (+12288380us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:53.175030 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:53.255515 (+ 80485us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:53.255566 (+ 51us) write_op.cc:435] Acquired schema lock
0812 19:07:53.255582 (+ 16us) tablet.cc:660] Decoding operations
0812 19:07:53.308634 (+ 53052us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:53.308730 (+ 96us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:53.308757 (+ 27us) tablet.cc:683] Acquiring locks for 18 operations
0812 19:07:53.310143 (+ 1386us) tablet.cc:699] Row locks acquired
0812 19:07:53.310180 (+ 37us) write_op.cc:260] PREPARE: finished
0812 19:07:53.310358 (+ 178us) write_op.cc:270] Start()
0812 19:07:53.310531 (+ 173us) write_op.cc:276] Timestamp: P: 1786561673310292 usec, L: 0
0812 19:07:53.310582 (+ 51us) op_driver.cc:331] REPLICATION: starting
0812 19:07:53.367822 (+ 57240us) log.cc:842] Serialized 7068 byte log entry
0812 19:07:56.628151 (+3260329us) op_driver.cc:444] REPLICATION: finished
0812 19:07:57.372075 (+743924us) write_op.cc:301] APPLY: starting
0812 19:07:57.372178 (+ 103us) tablet.cc:1365] starting BulkCheckPresence
0812 19:07:57.372535 (+ 357us) tablet.cc:1368] finished BulkCheckPresence
0812 19:07:57.372676 (+ 141us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:07:57.374710 (+ 2034us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:07:57.375123 (+ 413us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0812 19:07:57.375173 (+ 50us) write_op.cc:312] APPLY: finished
0812 19:07:57.375638 (+ 465us) log.cc:842] Serialized 125 byte log entry
0812 19:07:57.465553 (+ 89915us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:57.465741 (+ 188us) write_op.cc:454] Released schema lock
0812 19:07:57.465811 (+ 70us) write_op.cc:341] FINISH: Updating metrics
0812 19:07:57.466172 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:07:57.466204 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":33322,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":649620,"apply.run_cpu_time_us":11404,"apply.run_wall_time_us":94610,"num_ops":18,"prepare.queue_time_us":483723,"prepare.run_cpu_time_us":92898,"prepare.run_wall_time_us":2265471,"raft.queue_time_us":1631242,"raft.run_cpu_time_us":39795,"raft.run_wall_time_us":1356708,"replication_time_us":3161719,"thread_start_us":4759532,"threads_started":5,"wal-append.queue_time_us":1451942,"wal-append.run_cpu_time_us":53422,"wal-append.run_wall_time_us":3667147}]]}
I20260812 19:07:59.494037 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.494715 29457 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:59.495630 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.494452 1839 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.496706 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.497435 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.477401 29577 rpcz_store.cc:276] 0812 19:07:56.635971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.997163 (+2361192us) service_pool.cc:227] Handling call
0812 19:07:59.014792 (+ 17629us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:07:59.076565 (+ 61773us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:07:59.076593 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:07:59.103853 (+ 27260us) log.cc:842] Serialized 26 byte log entry
0812 19:07:59.185528 (+ 81675us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b48002301d8. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6ecb57f1 00007f9c6ecac8e7 00007f9c6ecac77a 00007f9c6eda095f 00007f9c6eda0627 00007f9c6edb4b2c 00007f9c6edb4ad8 00007f9c6edb4a58 00007f9c6edb4a13 00007f9c6edb3cbc 00007f9c6dd35be6
0812 19:07:59.216880 (+ 31352us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:07:59.216922 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.216994 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:07:59.397983 (+180989us) raft_consensus.cc:1671] finished
0812 19:07:59.398002 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.404651 (+ 6649us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4490,"raft.run_cpu_time_us":2874,"raft.run_wall_time_us":107383,"spinlock_wait_cycles":2554624,"thread_start_us":88638,"threads_started":1,"wal-append.queue_time_us":99629}
I20260812 19:07:59.497290 29457 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:07:59.508318 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.494736 29459 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:07:59.500275 29446 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.518373 29453 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260812 19:07:59.518674 29530 leader_election.cc:334] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.646s (SENT)
I20260812 19:07:59.519449 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.505029 29456 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:07:59.521242 29574 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:07:59.521303 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:59.522490 29326 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:59.523169 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2215) took 1628 ms. Trace:
I20260812 19:07:59.523454 29575 rpcz_store.cc:276] 0812 19:07:57.894981 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.022898 (+1127917us) service_pool.cc:227] Handling call
0812 19:07:59.044355 (+ 21457us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:07:59.081638 (+ 37283us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:07:59.276834 (+195196us) log.cc:842] Serialized 14 byte log entry
0812 19:07:59.522843 (+246009us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:07:59.522892 (+ 49us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:07:59.522945 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:07:59.523008 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:07:59.523110 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":160280,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":25798,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":330234,"threads_started":2}
I20260812 19:07:59.508961 29451 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:59.525549 1904 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.402815 1502 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.112s
I20260812 19:07:59.526955 1717 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:07:59.510676 29528 leader_election.cc:334] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.504s (SENT)
I20260812 19:07:59.533200 29528 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.524837 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.534631 1772 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:59.534942 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.535590 29446 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
W20260812 19:07:59.538645 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 50019.8us
I20260812 19:07:59.538890 29176 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:59.541879 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.542173 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.542770 29450 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:59.543073 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.543618 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.544389 29450 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.545070 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.546129 29176 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:59.548243 29176 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 1 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 19:07:59.502755 1515 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.551095 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.543691 29459 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
W20260812 19:07:59.554467 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 277133us
I20260812 19:07:59.545286 1971 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.558408 29450 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:59.559731 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.559842 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:59.561093 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 146010us
I20260812 19:07:59.561787 30623 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.563344 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.560477 29458 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:59.563621 29334 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260812 19:07:59.555224 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 86818.1us
I20260812 19:07:59.538480 1979 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:59.499118 1649 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:59.564805 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 54170.2us
I20260812 19:07:59.565413 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.567173 1973 raft_consensus.cc:2741] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260812 19:07:59.567878 29445 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:59.567605 1984 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:59.567062 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.567028 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.567546 29176 catalog_manager.cc:5495] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:07:59.568467 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.499022 1638 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.574250 29337 raft_consensus.cc:2354] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5.
I20260812 19:07:59.546054 1362 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:07:59.580510 1979 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.581449 1362 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.581634 29339 raft_consensus.cc:2354] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8.
I20260812 19:07:59.576752 29450 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.581208 1638 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.582183 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.582870 29445 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.572237 1998 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.583447 1998 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.583823 1998 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.556491 1717 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.586185 1717 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.586470 2006 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:59.588006 2006 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:07:59.589596 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.590519 1997 consensus_queue.cc:1046] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:59.507579 1478 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.571434 1967 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.603960 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.596100 1839 raft_consensus.cc:3032] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:59.597081 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.599085 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:37750 (request call id 964) took 1462 ms. Trace:
I20260812 19:07:59.601818 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.605306 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.611440 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.600124 1997 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:07:59.610527 29331 rpcz_store.cc:276] 0812 19:07:58.136832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.139066 (+ 2234us) service_pool.cc:227] Handling call
0812 19:07:59.591661 (+1452595us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003903d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6ed53c6e 00007f9c68456807 00007f9c68456792 00007f9c684566c1 00007f9c6845663c 00007f9c684558f2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:59.599003 (+ 7342us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2933370240}
I20260812 19:07:59.615305 1807 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.616369 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.619216 29446 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.617281 1478 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.573932 1649 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.621302 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.630041 30679 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:07:59.622130 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.624886 29453 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260812 19:07:59.631587 29337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2240) took 1349 ms (client timeout 1652 ms). Trace:
I20260812 19:07:59.629736 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.632491 29446 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:07:59.631956 29337 rpcz_store.cc:269] 0812 19:07:58.277633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.302722 (+ 25089us) service_pool.cc:227] Handling call
0812 19:07:59.617862 (+1315140us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b5400218254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:59.627048 (+ 9186us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1497480448}
I20260812 19:07:59.629372 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.621513 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.635905 29451 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:07:59.636575 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 72256.8us
I20260812 19:07:59.636687 1984 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:07:59.637499 29342 raft_consensus.cc:2396] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260812 19:07:59.641831 1991 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:07:59.642462 29342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2229) took 1501 ms (client timeout 1831 ms). Trace:
I20260812 19:07:59.642328 1991 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:07:59.642830 29342 rpcz_store.cc:269] 0812 19:07:58.140892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:58.274837 (+133945us) service_pool.cc:227] Handling call
0812 19:07:59.642266 (+1367429us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001504d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:07:59.642405 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3059204864}
I20260812 19:07:59.643975 29334 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:59.644143 1978 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.625764 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:59.647747 29528 leader_election.cc:334] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.646s (SENT)
I20260812 19:07:59.648452 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.620922 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.638736 29459 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:07:59.638028 29403 leader_election.cc:334] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 2.011s (SENT)
I20260812 19:07:59.652307 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.646430 29446 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:07:59.655102 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.606298 1839 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:07:59.660277 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.658393 29452 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:07:59.665377 1984 consensus_queue.cc:1046] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260812 19:07:59.651257 1772 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:07:59.666486 1984 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:59.666828 1984 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:07:59.667155 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49275.7us
W20260812 19:07:59.667866 29403 leader_election.cc:334] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 2.029s (SENT)
I20260812 19:07:59.667622 1984 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:59.668444 29403 leader_election.cc:334] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.648s (SENT)
I20260812 19:07:59.668633 1810 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:07:59.669382 1991 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.670564 1978 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.674798 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.675110 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.675675 29575 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:07:59.675918 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.676272 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.676491 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.676903 29567 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:59.677039 29574 raft_consensus.cc:2362] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5. Re-sending same reply.
I20260812 19:07:59.673862 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:07:59.628849 1478 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:59.678771 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:59.680274 29403 leader_election.cc:334] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.652s (SENT)
I20260812 19:07:59.693041 1807 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:07:59.693413 29459 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:07:59.695393 29403 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.695792 1984 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.696288 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:59.697461 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 93867.7us
I20260812 19:07:59.699204 29281 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:59.700397 29458 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:07:59.702056 1807 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:07:59.687551 1478 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:59.707821 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.708333 1478 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.708739 1986 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:07:59.719414 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.721661 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.723322 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.723943 29458 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.725250 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.725651 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.726081 30671 raft_consensus.cc:2379] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:07:59.727461 30681 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
I20260812 19:07:59.727555 29281 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.728751 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.723254 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:07:59.730038 29452 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:07:59.710726 1986 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:59.731995 1986 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:07:59.625506 1998 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.734565 1839 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.735898 1839 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:59.736181 1998 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.737213 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.738389 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.739276 30675 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:59.721208 1971 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:07:59.741735 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.723135 29446 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
W20260812 19:07:59.742827 29528 leader_election.cc:334] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.699s (SENT)
I20260812 19:07:59.743302 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.744343 1772 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:07:59.729882 29281 leader_election.cc:398] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
W20260812 19:07:59.746032 29528 leader_election.cc:334] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.739s (SENT)
I20260812 19:07:59.747051 29528 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.746804 29281 leader_election.cc:401] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260812 19:07:59.748924 29452 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:07:59.751350 29567 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:07:59.620146 2009 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.114s
I20260812 19:07:59.757266 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.768278 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:07:59.769318 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 66593.8us
I20260812 19:07:59.790457 30679 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:07:59.791385 29282 connection.cc:232] RPC call timeout handler was delayed by -1.7398028390016407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260812 19:07:59.791786 29282 connection.cc:232] RPC call timeout handler was delayed by -1.6543028389569372s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260812 19:07:59.792129 29282 connection.cc:232] RPC call timeout handler was delayed by -1.5841028389986604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260812 19:07:59.792429 29282 connection.cc:232] RPC call timeout handler was delayed by -1.4122028390411288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260812 19:07:59.792695 29282 connection.cc:232] RPC call timeout handler was delayed by -1.0279028390068561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2437s for a response
I20260812 19:07:59.793874 29282 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:59.794756 2001 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.795212 2001 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:59.795387 29458 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:07:59.795658 2001 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.799990 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.810848 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:59.812175 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:07:59.813479 29528 leader_election.cc:334] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.803s (SENT)
I20260812 19:07:59.813022 30623 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.809643 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.776912 1362 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.775754 1649 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:59.791839 1638 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:07:59.797266 1741 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:07:59.831933 2034 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260812 19:07:59.833431 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 61452.8us
I20260812 19:07:59.837782 29530 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.840996 2034 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:07:59.846793 29530 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.848129 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.848961 2034 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:59.849596 2034 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:59.850999 29530 leader_election.cc:398] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:07:59.851307 29530 leader_election.cc:401] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:07:59.853334 29530 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.854825 2046 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:59.854290 2034 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:07:59.856492 29530 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.821666 1984 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:07:59.857216 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.819357 1858 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:59.861703 1986 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.862062 1638 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.863812 1986 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.864020 1638 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.820948 1649 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:59.864825 1649 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
W20260812 19:07:59.866078 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 53123.5us
I20260812 19:07:59.868108 30623 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:07:59.871305 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.818769 1362 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:07:59.872973 1895 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:07:59.876123 1362 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.878625 2001 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:07:59.821672 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 156727us
I20260812 19:07:59.821861 30684 raft_consensus.cc:1226] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 2. Current term is 3. Ops: []
I20260812 19:07:59.885301 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:07:59.885701 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 72743.7us
I20260812 19:07:59.886672 2001 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.888993 1649 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.890089 1741 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:59.890604 29527 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.891579 1741 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:59.891780 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.892071 1741 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:07:59.892499 1741 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.892933 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.893270 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.893728 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.894323 29446 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:07:59.892457 29457 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:59.894963 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.895560 1649 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.897257 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.897883 29453 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
W20260812 19:07:59.884267 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 87227.8us
I20260812 19:07:59.897811 29530 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.892402 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.899842 29452 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.899114 29456 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:07:59.903362 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.895514 29451 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:07:59.903680 2045 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:07:59.874894 2042 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:07:59.905483 2042 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:59.906301 2042 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.907704 2042 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.910498 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.911139 29458 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
W20260812 19:07:59.911661 30625 leader_election.cc:334] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.591s (SENT)
I20260812 19:07:59.899780 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.915165 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.915509 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.916776 2003 consensus_queue.cc:1046] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:07:59.917912 1515 raft_consensus.cc:3032] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:07:59.918274 1515 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:07:59.921066 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 191293us
I20260812 19:07:59.922308 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.923429 1839 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:59.919210 1478 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.922943 29458 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:07:59.923329 2045 consensus_queue.cc:1046] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260812 19:07:59.925370 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.925061 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.926163 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.928063 2045 raft_consensus.cc:3032] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:07:59.926726 29281 leader_election.cc:398] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:59.925702 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:07:59.928994 29281 leader_election.cc:401] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260812 19:07:59.919040 1515 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.928763 29451 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:07:59.930064 1515 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:07:59.925314 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.930536 1839 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:07:59.931522 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.929351 29446 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:07:59.932288 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:07:59.932570 1967 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:07:59.932998 1967 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.933323 1967 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:59.933466 2009 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7.
I20260812 19:07:59.928074 29456 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:07:59.933787 1807 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.934195 1807 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:59.934594 1807 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.928776 2045 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:07:59.936206 2045 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.926333 1478 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:07:59.934240 2009 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:07:59.939296 2009 raft_consensus.cc:686] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:59.940383 2009 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.943429 29282 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:59.944800 1991 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:07:59.945338 1991 raft_consensus.cc:686] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:59.946024 1638 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:07:59.946265 1991 consensus_queue.cc:237] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.947805 1638 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 19:07:59.928301 30625 leader_election.cc:334] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.061s (SENT)
I20260812 19:07:59.949635 30625 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.923795 29530 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.938642 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.945348 29281 leader_election.cc:398] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:07:59.937703 2045 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:07:59.954478 1717 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:07:59.954888 1717 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:07:59.955210 1717 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:07:59.952857 1638 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:07:59.958303 1638 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:07:59.958803 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.959621 1904 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:07:59.959930 29399 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.960059 1904 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:07:59.959193 1638 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.964476 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.960526 1904 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:07:59.967762 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.970466 29282 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:59.973274 29282 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:07:59.977453 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:07:59.978179 30623 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:07:59.978492 29575 raft_consensus.cc:2354] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8.
I20260812 19:07:59.980456 30684 raft_consensus.cc:2354] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8.
I20260812 19:07:59.984725 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.986390 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:59.969193 1772 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:07:59.988955 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:59.966502 1839 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:07:59.990494 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.992697 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:07:59.993312 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:07:59.993319 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.994130 29334 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:07:59.937786 1362 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:07:59.995776 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:07:59.981369 1986 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
W20260812 19:07:59.996475 29528 leader_election.cc:334] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.959s (SENT)
I20260812 19:07:59.985445 2006 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:07:59.997457 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.002588 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.003232 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.998274 29567 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:00.001144 29528 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:07:59.987200 2003 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:00.004335 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:07:59.982131 2034 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:00.002707 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.001010 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:00.009119 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.009356 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.010094 29334 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:00.010756 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:00.011051 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 82771.2us
I20260812 19:08:00.012494 29403 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:00.009137 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.015429 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.017151 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:00.017700 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:00.018491 29459 raft_consensus.cc:2396] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:00.019459 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:07:59.998755 1638 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:00.056375 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.057063 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.058234 29572 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:00.019272 29446 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:00.060282 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:00.060884 29451 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:00.064535 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 48831.2us
W20260812 19:08:00.068328 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55950.2us
I20260812 19:08:00.002411 30681 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:00.077746 29458 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.016191 1649 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:00.095001 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 101474us
I20260812 19:08:00.059943 1986 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.065479 2003 raft_consensus.cc:686] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:00.096287 29528 leader_election.cc:334] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.069s (SENT)
I20260812 19:08:00.104952 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.099087 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.105824 29567 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.078744 2006 raft_consensus.cc:686] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:00.108064 2006 consensus_queue.cc:237] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.016412 1741 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:00.121511 29528 leader_election.cc:334] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.117s (SENT)
I20260812 19:08:00.083921 2034 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:08:00.123282 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.106976 2046 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:00.131983 29451 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:00.079010 2056 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:00.141839 29281 leader_election.cc:398] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8.
I20260812 19:08:00.142251 29281 leader_election.cc:401] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:08:00.092027 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:00.139684 29572 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:00.136385 1362 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:00.143380 30625 leader_election.cc:334] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.295s (SENT)
I20260812 19:08:00.150538 29575 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.174973 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:00.178872 30625 leader_election.cc:334] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.355s (SENT)
I20260812 19:08:00.179364 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:00.181147 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 59673.7us
I20260812 19:08:00.081471 1807 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.186090 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.186789 30671 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:00.190536 29331 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:00.197531 29458 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:00.104564 1649 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.152418 2034 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:00.204550 29567 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
W20260812 19:08:00.192000 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 199387us
I20260812 19:08:00.205389 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:00.207365 30675 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:00.210386 2066 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.195047 1858 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 8
I20260812 19:08:00.245153 2003 consensus_queue.cc:237] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:00.242638 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 63790.1us
I20260812 19:08:00.244069 1986 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.245985 1810 consensus_queue.cc:1046] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260812 19:08:00.301649 30671 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:00.300699 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:00.317726 29575 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:00.347360 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.347555 29572 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:08:00.364946 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.210928 2084 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.368940 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.317684 2066 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:00.371021 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 229330us
I20260812 19:08:00.371853 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:00.360225 2057 log.cc:924] Time spent T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.263s user 0.003s sys 0.000s
I20260812 19:08:00.347573 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:00.382665 1362 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:00.382980 30674 raft_consensus.cc:2396] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:00.385798 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:00.387799 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.548321 29572 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:00.551095 29331 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:00.387481 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:00.554306 29176 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:53840 (request call id 290) took 13532 ms (client timeout 15000 ms). Trace:
I20260812 19:08:00.377044 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.368479 2084 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:00.373430 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.559278 2084 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.374336 2066 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.366294 1967 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:00.477986 2069 log.cc:924] Time spent T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.129s user 0.000s sys 0.005s
I20260812 19:08:00.479408 2073 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.561801 29577 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260812 19:08:00.481874 2059 log.cc:924] Time spent T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.353s user 0.003s sys 0.003s
I20260812 19:08:00.570971 29337 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:00.484784 1502 raft_consensus.cc:3032] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:00.571651 29572 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:00.571890 1502 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:00.487509 1998 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:00.490238 29579 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
W20260812 19:08:00.490688 1904 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.504s user 0.012s sys 0.000s
W20260812 19:08:00.491060 2042 consensus_meta.cc:287] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.583s user 0.010s sys 0.003s
I20260812 19:08:00.494398 1741 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.366029 2062 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:00.500243 30623 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.580927 29282 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:00.505005 29342 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:00.508749 1807 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.513705 29575 raft_consensus.cc:1261] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260812 19:08:00.515156 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 214533us
I20260812 19:08:00.587102 1478 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.588215 1478 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:00.588564 1478 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.588778 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.590032 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:00.580986 2062 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:00.590677 2062 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:00.591651 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.591796 29339 raft_consensus.cc:1226] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:00.524606 1858 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8.
I20260812 19:08:00.527388 29569 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
W20260812 19:08:00.527918 1717 consensus_meta.cc:287] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.572s user 0.008s sys 0.015s
I20260812 19:08:00.528802 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.598366 30686 raft_consensus.cc:2379] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:00.598084 1717 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.529673 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.563269 30671 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 3: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:00.565933 29338 raft_consensus.cc:1226] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:00.569588 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.569979 2073 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:00.570582 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:00.574741 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:00.575501 29403 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:00.575716 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 190000us
I20260812 19:08:00.604308 1717 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.608722 1858 consensus_queue.cc:1046] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:08:00.609975 30671 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:00.610059 1638 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:00.610672 1638 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.611577 1638 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.612867 1638 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.621651 29281 leader_election.cc:398] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8.
I20260812 19:08:00.623049 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.625010 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.625730 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:00.554785 29176 rpcz_store.cc:269] 0812 19:07:47.021659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:47.036179 (+ 14520us) service_pool.cc:227] Handling call
0812 19:08:00.554230 (+13518051us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:07:59.823976 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:07:59.824655 (+ 679us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:07:59.824710 (+ 55us) write_op.cc:435] Acquired schema lock
0812 19:07:59.824728 (+ 18us) tablet.cc:660] Decoding operations
0812 19:07:59.825626 (+ 898us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:07:59.825742 (+ 116us) write_op.cc:641] Partition lock acquired for write op
0812 19:07:59.825767 (+ 25us) tablet.cc:683] Acquiring locks for 6 operations
0812 19:07:59.826394 (+ 627us) tablet.cc:699] Row locks acquired
0812 19:07:59.826429 (+ 35us) write_op.cc:260] PREPARE: finished
0812 19:07:59.826613 (+ 184us) write_op.cc:270] Start()
0812 19:07:59.826751 (+ 138us) write_op.cc:276] Timestamp: P: 1786561679826556 usec, L: 0
0812 19:07:59.826790 (+ 39us) op_driver.cc:331] REPLICATION: starting
0812 19:07:59.827454 (+ 664us) log.cc:842] Serialized 2452 byte log entry
0812 19:08:00.209346 (+381892us) op_driver.cc:444] REPLICATION: finished
0812 19:08:00.514583 (+305237us) write_op.cc:301] APPLY: starting
0812 19:08:00.514694 (+ 111us) tablet.cc:1365] starting BulkCheckPresence
0812 19:08:00.514898 (+ 204us) tablet.cc:1368] finished BulkCheckPresence
0812 19:08:00.515110 (+ 212us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:08:00.516176 (+ 1066us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:08:00.516535 (+ 359us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0812 19:08:00.516584 (+ 49us) write_op.cc:312] APPLY: finished
0812 19:08:00.516983 (+ 399us) log.cc:842] Serialized 53 byte log entry
0812 19:08:00.517228 (+ 245us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:00.517356 (+ 128us) write_op.cc:454] Released schema lock
0812 19:08:00.517418 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0812 19:08:00.519077 (+ 1659us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:00.519117 (+ 40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":584787,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":304843,"apply.run_cpu_time_us":3517,"apply.run_wall_time_us":45194,"num_ops":6,"prepare.queue_time_us":30742,"prepare.run_cpu_time_us":4575,"prepare.run_wall_time_us":22056,"raft.queue_time_us":377592,"raft.run_cpu_time_us":29415,"raft.run_wall_time_us":165712,"replication_time_us":382454,"thread_start_us":325539,"threads_started":4,"wal-append.queue_time_us":1368}]]}
I20260812 19:08:00.585799 29177 catalog_manager.cc:5495] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 5 to 9, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 9 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
W20260812 19:08:00.594007 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 93853.5us
I20260812 19:08:00.631036 29399 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.601285 2073 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.603538 29334 raft_consensus.cc:2379] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:00.602758 29332 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:00.583079 29282 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:00.632751 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49752us
I20260812 19:08:00.633387 29282 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:00.578764 2052 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:00.637055 30686 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:08:00.639199 29177 catalog_manager.cc:5495] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:08:00.641288 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.643072 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:00.645141 29459 raft_consensus.cc:1226] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:08:00.627223 2100 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:00.628814 1722 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:00.649587 30684 raft_consensus.cc:2379] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:00.651364 29577 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:00.652182 2100 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:08:00.653167 2100 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:00.629494 2102 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:00.575220 1502 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.653543 2100 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:00.653899 2102 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:00.628679 2096 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:00.578955 1904 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.636624 1717 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:00.655012 29177 catalog_manager.cc:5495] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:08:00.638643 1807 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:00.660560 1722 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:00.661233 30674 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260812 19:08:00.662940 29452 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:00.645072 1986 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:00.641433 2052 raft_consensus.cc:686] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.658239 1717 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:00.634547 1839 consensus_queue.cc:1046] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260812 19:08:00.682789 1839 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:00.682900 2071 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:00.683547 2068 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:00.683326 1839 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.654702 1502 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:00.664067 1807 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:00.678813 1722 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:00.689321 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 48105.9us
I20260812 19:08:00.659384 2102 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:00.664222 1967 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.692246 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:00.692255 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44704.8us
I20260812 19:08:00.696266 2117 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:00.696638 2117 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:00.696918 2117 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:00.658636 1997 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260812 19:08:00.686769 1986 raft_consensus.cc:686] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.690608 1807 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:00.708739 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:00.684580 1717 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:00.691291 1945 raft_consensus.cc:3032] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:00.705065 1904 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.728204 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:08:00.692380 2071 raft_consensus.cc:686] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.690857 2068 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:00.746758 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.743047 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:00.743891 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 110574us
I20260812 19:08:00.714910 29177 catalog_manager.cc:5495] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:00.724722 1717 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:00.726668 1945 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:00.749541 1945 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.750733 1945 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:00.752452 1986 consensus_queue.cc:237] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.727099 2066 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.754424 2068 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.755803 2066 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.756376 2066 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:00.757274 1478 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.729265 2052 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.748759 1967 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:00.748700 29177 catalog_manager.cc:5495] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:08:00.759456 29574 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260812 19:08:00.776819 29528 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.781507 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:00.789685 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:00.784125 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 208694us
I20260812 19:08:00.788592 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:00.816434 29580 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:00.822499 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.822806 29177 catalog_manager.cc:5495] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 1 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:00.824339 29177 catalog_manager.cc:5495] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 3 to 4, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 4 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:08:00.796433 2117 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.828289 2117 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.831204 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:00.832445 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 55735.8us
I20260812 19:08:00.840312 29528 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.841629 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.842140 2063 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:00.842692 2071 consensus_queue.cc:237] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.726331 1968 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260812 19:08:00.791533 1967 raft_consensus.cc:686] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:08:00.855909 29399 connection.cc:232] RPC call timeout handler was delayed by -1.2595255591440946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260812 19:08:00.856344 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1920255590230227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1622s for a response
W20260812 19:08:00.849233 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 106266us
I20260812 19:08:00.849851 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.855034 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.795441 1741 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:00.859136 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.829108 1649 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.784556 2006 consensus_queue.cc:1035] T 62647baf8dbe49b9b57e1f296bfa85e6 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260812 19:08:00.844607 2118 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:08:00.861598 29177 catalog_manager.cc:5495] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:08:00.843849 2062 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.868145 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:00.867811 2056 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.870666 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:00.856623 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1686255591921508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260812 19:08:00.877233 29177 catalog_manager.cc:5495] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 4, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:08:00.878284 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1623255591839552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260812 19:08:00.878623 29399 connection.cc:232] RPC call timeout handler was delayed by -1.1299255590420216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260812 19:08:00.878896 29399 connection.cc:232] RPC call timeout handler was delayed by -1.0993255591019988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260812 19:08:00.849195 2092 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260812 19:08:00.880191 29399 connection.cc:232] RPC call timeout handler was delayed by -1.0903255590237677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260812 19:08:00.881968 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:00.891402 29399 connection.cc:232] RPC call timeout handler was delayed by -1.0192255591973662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
W20260812 19:08:00.894677 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 99896.3us
I20260812 19:08:00.894304 29177 catalog_manager.cc:5495] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:00.916157 29446 raft_consensus.cc:2379] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:00.862533 2063 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:00.922371 29399 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.875180 1839 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.934598 29177 catalog_manager.cc:5495] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:00.859141 1895 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260812 19:08:00.946301 1722 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:00.946906 1722 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:00.947415 1722 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:00.948990 2062 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:00.940328 1904 raft_consensus.cc:2741] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260812 19:08:00.954352 1478 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.955291 29567 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:08:00.887684 2092 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:00.930583 2063 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:00.969115 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 318295us
W20260812 19:08:00.969718 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 47454.4us
I20260812 19:08:00.973526 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:00.974248 29569 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:00.980581 1895 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:08:00.983539 2084 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:00.984221 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 358272us
I20260812 19:08:00.985008 29281 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.986699 1717 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.988751 1717 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:00.970103 1478 consensus_queue.cc:1046] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260812 19:08:00.993728 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 759) took 1011 ms. Trace:
I20260812 19:08:00.994846 1967 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:00.995831 29399 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.997051 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:00.997934 2125 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:00.998416 2125 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:00.998777 2125 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.002669 29399 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.976073 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:01.003912 2006 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:01.004283 2006 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:01.004602 2006 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:01.007313 1989 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:00.961987 2092 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:01.010746 1362 raft_consensus.cc:3032] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:08:01.011113 1362 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:01.011924 1362 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:00.971107 2063 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:00.994349 2042 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:01.014696 2042 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:01.015115 2042 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:01.014134 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:00.994633 29580 rpcz_store.cc:276] 0812 19:07:59.982629 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:00.001558 (+ 18929us) service_pool.cc:227] Handling call
0812 19:08:00.993554 (+991996us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400156654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:00.993672 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1195784064}
I20260812 19:08:01.019335 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:01.020502 29281 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:01.021900 30623 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:00.880188 2056 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:01.026118 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55691.7us
I20260812 19:08:01.041898 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:01.042565 29575 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 5
W20260812 19:08:01.048452 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 45872.1us
I20260812 19:08:01.048213 29567 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:01.048213 29331 raft_consensus.cc:1261] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:08:00.959787 1839 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260812 19:08:01.080641 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:01.038298 1997 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election won for term 7
W20260812 19:08:01.083060 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 201189us
I20260812 19:08:01.142663 29457 raft_consensus.cc:1226] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:01.105819 29575 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:01.026980 2123 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:01.023896 1858 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260812 19:08:01.152655 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 220541us
I20260812 19:08:01.157313 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:01.037729 2097 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:08:01.164738 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:01.165407 30671 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:01.046581 2056 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.084949 1904 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:01.084328 1717 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:01.167709 29530 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:01.169484 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:01.276446 29458 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.079371 2118 raft_consensus.cc:686] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:01.078440 1515 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260812 19:08:01.078853 1502 consensus_queue.cc:1046] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
W20260812 19:08:01.280232 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 409666us
I20260812 19:08:01.322887 29452 raft_consensus.cc:1261] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260812 19:08:01.046353 1638 consensus_queue.cc:1046] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260812 19:08:01.326056 29331 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "005defc807dc47d881add64226efb7ce"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:01.087675 2047 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.190s
I20260812 19:08:01.327126 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:01.327256 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:01.324297 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 266316us
I20260812 19:08:01.330979 29528 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:01.332329 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:01.334367 29331 raft_consensus.cc:595] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:01.326786 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:01.341536 30686 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:01.341580 29331 raft_consensus.cc:603] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:01.338598 1649 consensus_queue.cc:1046] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 19:08:01.274848 1717 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:01.416394 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 328137us
I20260812 19:08:01.282521 2117 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:01.425675 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:01.400182 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:01.466828 29339 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:01.399056 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:01.339030 1638 raft_consensus.cc:3032] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:01.457264 2148 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.245558 1998 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.179s
I20260812 19:08:01.331032 1978 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:01.517714 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 350046us
I20260812 19:08:01.523078 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:01.467639 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:01.524312 29453 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260812 19:08:01.400944 2073 consensus_meta.cc:287] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.761s user 0.007s sys 0.003s
W20260812 19:08:01.525326 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 444741us
I20260812 19:08:01.580854 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:01.567488 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 115510us
I20260812 19:08:01.569484 29446 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:01.577205 29458 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:01.243808 2127 consensus_queue.cc:1035] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.204s
I20260812 19:08:01.464682 1717 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.574240 29331 raft_consensus.cc:2362] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8. Re-sending same reply.
I20260812 19:08:01.584396 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:01.516065 1967 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s user 0.033s sys 0.027s
W20260812 19:08:01.615475 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.753s user 0.003s sys 0.006s
I20260812 19:08:01.511536 1638 raft_consensus.cc:729] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:01.640970 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s user 0.003s sys 0.006s
W20260812 19:08:01.583554 2102 consensus_meta.cc:287] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.881s user 0.010s sys 0.000s
W20260812 19:08:01.642433 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 311507us
W20260812 19:08:01.641891 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 779736us
I20260812 19:08:01.644686 29527 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:01.645596 29528 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:01.636902 2095 log.cc:924] Time spent T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.119s user 0.000s sys 0.001s
I20260812 19:08:01.658083 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:01.635852 1807 consensus_meta.cc:287] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.905s user 0.005s sys 0.006s
I20260812 19:08:01.628187 2066 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:01.585638 1978 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.721158 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 75637.8us
I20260812 19:08:01.756925 29528 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:01.741438 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 132182us
I20260812 19:08:01.650203 2152 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.639138 2062 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:08:01.655860 1839 raft_consensus.cc:3032] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
W20260812 19:08:01.661513 2122 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s user 0.008s sys 0.000s
I20260812 19:08:01.457482 2118 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.734344 2142 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.394s user 0.000s sys 0.001s
I20260812 19:08:01.624650 2073 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.737399 1967 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:08:01.769814 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 111841us
I20260812 19:08:01.567508 2148 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.739817 2176 raft_consensus.cc:2741] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260812 19:08:01.818050 29281 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:01.815433 29399 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:01.825899 2092 consensus_meta.cc:287] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.816s user 0.006s sys 0.003s
I20260812 19:08:01.838546 2092 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.839864 2100 consensus_meta.cc:287] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.185s user 0.007s sys 0.006s
I20260812 19:08:01.840809 2144 consensus_queue.cc:1046] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.196s
I20260812 19:08:01.841621 29574 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:01.813407 1967 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:01.844240 1967 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:01.845257 2042 consensus_meta.cc:287] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.829s user 0.009s sys 0.001s
I20260812 19:08:01.831270 29578 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260812 19:08:01.832816 2125 consensus_meta.cc:287] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.834s user 0.003s sys 0.006s
I20260812 19:08:01.834465 2123 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260812 19:08:01.833554 29580 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260812 19:08:01.653689 2102 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.766362 2122 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s user 0.010s sys 0.000s
I20260812 19:08:01.766546 1839 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:01.835978 2135 log.cc:924] Time spent T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.185s user 0.000s sys 0.000s
W20260812 19:08:01.847414 2006 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.842s user 0.008s sys 0.001s
I20260812 19:08:01.835932 2191 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:01.667441 1807 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.866786 2084 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:01.859242 1722 consensus_meta.cc:287] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.911s user 0.010s sys 0.000s
I20260812 19:08:01.893148 2006 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.893924 1722 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.858966 2062 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:01.894450 1839 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.851368 1998 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.895763 1998 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.865967 2042 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.897269 2056 consensus_meta.cc:287] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.622s user 0.005s sys 0.000s
I20260812 19:08:01.898092 2188 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.898643 2188 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.771751 2152 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:01.850230 2063 consensus_meta.cc:287] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.798s user 0.006s sys 0.003s
I20260812 19:08:01.901702 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:01.901865 2063 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.902413 2152 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.903314 2042 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.866828 2176 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:08:01.905992 1362 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:01.842481 29339 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:01.907569 2092 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.907840 29580 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:01.908324 30674 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:01.908816 2063 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.865736 2125 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.911182 2125 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.915304 2102 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.917397 1741 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:01.918560 2175 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:01.919111 2175 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.840317 2100 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:01.879395 1986 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:01.921839 1986 raft_consensus.cc:686] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:01.922569 2191 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:01.923065 2191 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.922747 1986 consensus_queue.cc:237] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:01.909044 2172 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260812 19:08:01.867794 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 53088.3us
I20260812 19:08:01.928579 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:01.929575 2062 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:01.902588 30671 raft_consensus.cc:2396] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:01.917831 1741 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:01.937108 1741 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:01.879108 1717 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.941066 29459 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
W20260812 19:08:01.943869 2150 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.288s user 0.003s sys 0.000s
I20260812 19:08:01.947429 1807 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.947688 2146 raft_consensus.cc:3032] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:01.948184 2146 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:01.853824 1638 consensus_queue.cc:260] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:01.951365 2073 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:01.949077 2146 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:01.955962 2006 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:01.963771 1722 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:01.965023 1638 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:01.966468 29457 raft_consensus.cc:1226] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
W20260812 19:08:01.967478 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 210761us
I20260812 19:08:01.962838 2175 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:01.971138 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:01.972205 30625 leader_election.cc:334] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.528s (SENT)
I20260812 19:08:01.973706 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:01.965396 1978 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:01.984146 30625 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:01.970039 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:01.967785 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 247212us
I20260812 19:08:01.984355 30671 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:01.990628 30675 raft_consensus.cc:1261] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260812 19:08:01.979532 1997 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.733s user 0.024s sys 0.011s
I20260812 19:08:01.966667 2131 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.998224 29528 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:01.998436 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:02.003444 30685 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260812 19:08:02.003767 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:02.001152 30625 leader_election.cc:334] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.536s (SENT)
I20260812 19:08:02.006875 2200 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.007511 2148 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:02.007404 2200 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.008026 2117 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:08:01.991536 2146 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:01.897703 2056 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.009110 29528 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:02.010130 2212 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:02.010339 2056 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.010871 2056 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:08:02.011220 2056 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.006916 1998 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:02.011588 2056 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:02.012111 29528 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:02.013347 29528 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:02.013728 29576 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:02.014513 30681 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:02.015080 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:02.022285 29399 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:01.986384 1717 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:02.005893 30625 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.766s (SENT) [suppressed 83 similar messages]
I20260812 19:08:02.012156 2194 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:01.994009 1807 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260812 19:08:02.013689 1997 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.826s user 0.024s sys 0.011s
I20260812 19:08:02.028800 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:02.017670 2175 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.043792 29399 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:02.013278 2092 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:02.044865 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 772) took 1183 ms. Trace:
I20260812 19:08:02.028165 2117 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:02.022001 2176 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:02.026661 2073 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:02.029043 2042 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:02.019279 2214 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:02.023142 2100 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:02.023065 2212 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:02.028544 2125 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:02.042243 1997 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:02.050544 2145 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.590s user 0.000s sys 0.008s
I20260812 19:08:02.022603 2211 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:02.060485 29281 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:02.061736 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 778) took 1109 ms. Trace:
I20260812 19:08:02.062700 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:02.038383 2152 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.070628 30671 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:08:02.048358 29576 rpcz_store.cc:276] 0812 19:08:00.861460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:00.871962 (+ 10502us) service_pool.cc:227] Handling call
0812 19:08:00.872322 (+ 360us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.018987 (+1146665us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:02.044636 (+ 25649us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540012be54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:02.044804 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2624608128}
I20260812 19:08:02.098246 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:02.019111 1967 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.041381 1807 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:02.081943 29446 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:02.048012 1717 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:02.085150 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.095664 29574 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:02.097867 29579 rpcz_store.cc:276] 0812 19:08:00.952016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:00.954039 (+ 2023us) service_pool.cc:227] Handling call
0812 19:08:00.954341 (+ 302us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:01.325468 (+371127us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:01.325522 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:01.326894 (+ 1372us) log.cc:842] Serialized 26 byte log entry
0812 19:08:01.944042 (+617148us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b18003b2c60. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c69a9d82c 00007f9c66ed49dc 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb 00007f9c69a95063 00007f9c69a95411
0812 19:08:01.944217 (+ 175us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:02.050771 (+106554us) log.cc:842] Serialized 14 byte log entry
0812 19:08:02.051014 (+ 243us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:02.051107 (+ 93us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:02.051155 (+ 48us) raft_consensus.cc:1671] finished
0812 19:08:02.061418 (+ 10263us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:02.061674 (+ 256us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":571009,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":81024,"spinlock_wait_cycles":39319808,"thread_start_us":598430,"threads_started":3,"wal-append.queue_time_us":386003}
I20260812 19:08:02.100342 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.100617 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.067348 2214 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.018445 2131 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.164064 29335 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:02.107292 2100 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.039867 1515 consensus_queue.cc:1035] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260812 19:08:02.107774 29403 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:02.108000 29569 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260812 19:08:02.173496 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:02.172737 30685 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:02.174590 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:02.109764 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 47146us
I20260812 19:08:02.110275 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:02.110997 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:02.107199 29453 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:02.055711 2176 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:02.063761 2073 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.148965 2125 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:02.150509 29342 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:02.151686 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 91300us
I20260812 19:08:02.152192 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:02.231493 30684 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:02.066550 2042 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.065412 2175 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.234022 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:02.239435 29528 leader_election.cc:334] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.501s (SENT)
I20260812 19:08:02.239867 29528 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:02.106541 2207 consensus_queue.cc:1046] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
W20260812 19:08:02.159734 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49331.6us
W20260812 19:08:02.096645 2145 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.674s user 0.000s sys 0.009s
I20260812 19:08:02.256847 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.165086 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.164103 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.063200 2062 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260812 19:08:02.226569 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 58570.7us
I20260812 19:08:02.177178 29576 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:02.174150 30675 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:02.054572 2047 consensus_queue.cc:1046] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260812 19:08:02.096727 2092 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:02.148489 1807 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:02.228123 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 120420us
W20260812 19:08:02.227558 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 221716us
I20260812 19:08:02.271636 29403 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:02.226735 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 58759.6us
I20260812 19:08:02.060232 2068 consensus_queue.cc:1046] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260812 19:08:02.159157 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:02.063372 2122 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260812 19:08:02.159703 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:02.234702 29458 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:08:02.236713 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.237125 29334 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:02.274062 29574 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:02.163932 2214 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:02.244745 29457 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260812 19:08:02.246330 29342 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:02.113135 2118 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.281332 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.282018 29572 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:02.249450 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:02.112906 2152 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.282298 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.283275 29457 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:08:02.284163 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.258890 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:02.263289 29528 leader_election.cc:334] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.556s (SENT)
I20260812 19:08:02.112789 2003 consensus_queue.cc:1046] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.124s
I20260812 19:08:02.044885 2194 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.235478 2073 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:02.267334 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:02.267881 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:02.268603 29453 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260812 19:08:02.266520 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.294309 29445 raft_consensus.cc:2379] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:02.297435 29332 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:08:02.235292 1717 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:02.169605 1722 consensus_queue.cc:1046] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.085s
I20260812 19:08:02.227788 2176 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:02.241918 2042 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:02.269207 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.293668 29577 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:02.303357 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.304633 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.270598 30684 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:02.305835 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 871) took 1162 ms. Trace:
I20260812 19:08:02.090701 2212 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.250137 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.285301 29446 raft_consensus.cc:2379] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:02.285188 29528 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:02.282030 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.316286 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.264127 2062 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:02.290596 2118 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.286089 1820 consensus_queue.cc:1046] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
W20260812 19:08:02.339741 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 69334.3us
I20260812 19:08:02.243062 2100 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:02.300025 2216 consensus_queue.cc:1046] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260812 19:08:02.299901 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.344273 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.291412 2148 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.317080 29458 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:02.318648 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.306150 30684 rpcz_store.cc:276] 0812 19:08:01.143644 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:01.144224 (+ 580us) service_pool.cc:227] Handling call
0812 19:08:02.152031 (+1007807us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:02.305780 (+153749us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":11346688}
I20260812 19:08:02.320389 29527 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:02.287278 2084 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.338927 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.292943 2117 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:02.278402 2123 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260812 19:08:02.313277 29567 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:08:02.315073 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.377555 29446 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:02.348846 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.299904 2213 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.354430 29403 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:02.356078 29574 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:02.333186 2145 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.360368 29445 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:08:02.367381 2125 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:08:02.314003 2144 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:02.373660 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.269340 2097 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260812 19:08:02.376019 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 91676.3us
I20260812 19:08:02.334218 2062 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:02.278090 1997 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:02.338399 30625 leader_election.cc:334] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.770s (SENT)
I20260812 19:08:02.358930 29572 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:02.282114 2092 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.333909 1362 consensus_queue.cc:1046] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260812 19:08:02.317476 1989 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.150s
I20260812 19:08:02.437255 2084 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:02.773932 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.774708 29446 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:02.772092 2232 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:02.775490 2232 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:02.374743 1722 raft_consensus.cc:3032] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:02.776537 1722 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:02.776891 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.778069 29446 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:02.778496 29456 raft_consensus.cc:2379] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:08:02.446491 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 126161us
I20260812 19:08:02.783174 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.784550 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:02.788837 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.791045 29572 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:02.797147 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.798274 29574 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:08:02.802712 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 539426us
W20260812 19:08:02.804200 29528 leader_election.cc:334] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.218s (SENT)
I20260812 19:08:02.805506 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:02.813441 29458 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:02.816699 29399 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:02.818645 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.819890 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:02.436167 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:02.338804 2217 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:02.436061 2171 log.cc:924] Time spent T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.095s user 0.000s sys 0.000s
W20260812 19:08:02.736754 2146 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.722s user 0.004s sys 0.006s
W20260812 19:08:02.842836 1717 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.503s user 0.038s sys 0.058s
I20260812 19:08:02.856175 1717 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:02.857132 1717 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.839263 29446 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:02.858655 1820 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.858647 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.859802 29446 raft_consensus.cc:2354] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:08:02.859290 1820 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.860524 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.861222 29446 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:02.848202 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.862336 29445 raft_consensus.cc:2354] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9.
I20260812 19:08:02.777490 2232 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:02.728621 2131 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.745335 2200 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.777376 1722 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:02.378947 2186 log.cc:924] Time spent T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260812 19:08:02.854138 2004 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.839188 2145 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.855378 2118 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:02.774712 2092 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.437788 2216 consensus_queue.cc:1046] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.553s
I20260812 19:08:02.851191 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:02.861868 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:03.197878 29456 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9.
I20260812 19:08:02.350063 2211 consensus_queue.cc:1046] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.537s
I20260812 19:08:02.371078 2148 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:02.771898 2210 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:02.810691 2084 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:02.906268 2228 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.438s user 0.001s sys 0.000s
I20260812 19:08:03.001487 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:03.196623 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 858202us
I20260812 19:08:02.906152 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:03.182899 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2153) took 1103 ms. Trace:
W20260812 19:08:03.012053 2191 consensus_meta.cc:287] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.070s user 0.002s sys 0.012s
W20260812 19:08:02.838476 2209 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.005s sys 0.002s
W20260812 19:08:02.864924 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 45343.6us
I20260812 19:08:03.181862 29458 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:03.202742 29446 raft_consensus.cc:2379] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:03.231258 29450 raft_consensus.cc:2354] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:08:02.375039 2199 raft_consensus.cc:2741] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260812 19:08:02.512665 1967 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:03.207891 2232 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:02.438426 2117 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:02.797586 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 870) took 1601 ms. Trace:
W20260812 19:08:03.209013 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 864544us
I20260812 19:08:03.212906 2235 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:03.213649 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:02.908879 1362 raft_consensus.cc:3032] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
W20260812 19:08:02.993199 2056 consensus_meta.cc:287] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.981s user 0.010s sys 0.001s
W20260812 19:08:03.215591 29402 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.560s user 0.045s sys 0.037s
I20260812 19:08:03.216816 29451 rpcz_store.cc:276] 0812 19:08:02.079282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.079388 (+ 106us) service_pool.cc:227] Handling call
0812 19:08:02.107910 (+ 28522us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.382692 (+274782us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.382725 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:02.382765 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0812 19:08:02.382794 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.182593 (+799799us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002ff8d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:03.182650 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:03.182826 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":631505792}
I20260812 19:08:03.258806 29450 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260812 19:08:03.218317 29445 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
W20260812 19:08:03.259245 29402 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.726s user 0.045s sys 0.037s
I20260812 19:08:02.810382 2213 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.219838 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:03.220656 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:02.906832 2244 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:02.906994 2230 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 19:08:03.194439 2146 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:03.182195 2215 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:03.191949 2194 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:03.214398 1741 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.277s user 0.008s sys 0.003s
I20260812 19:08:02.893537 1820 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.900046 2217 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:02.916288 2207 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:03.239473 29335 raft_consensus.cc:595] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:03.239564 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:03.193079 2212 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:02.907799 2218 log.cc:924] Time spent T 00000000000000000000000000000000 P f47470148eaf49acb3f5b4813eb35f3e: Append to log took a long time: real 0.558s user 0.000s sys 0.001s
I20260812 19:08:03.362982 1741 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:03.204136 2210 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:02.862505 2188 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:03.248824 29452 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.910s user 0.060s sys 0.078s
W20260812 19:08:03.390232 29452 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.052s user 0.060s sys 0.078s
I20260812 19:08:03.389281 2210 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.390969 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2152) took 1312 ms. Trace:
I20260812 19:08:03.338750 2244 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.214493 2152 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:02.823951 2144 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.391320 29452 rpcz_store.cc:276] 0812 19:08:02.079013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.079125 (+ 112us) service_pool.cc:227] Handling call
0812 19:08:02.148928 (+ 69803us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.149152 (+ 224us) raft_consensus.cc:1520] Early marking committed up to index 9
0812 19:08:02.338106 (+188954us) log.cc:842] Serialized 14 byte log entry
0812 19:08:03.390632 (+1052526us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:03.390683 (+ 51us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:03.390731 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.390800 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:03.390917 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":322346,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"thread_start_us":1162670,"threads_started":2}
W20260812 19:08:03.230304 2209 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.940s user 0.014s sys 0.002s
I20260812 19:08:02.873720 29572 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
W20260812 19:08:03.218740 30625 leader_election.cc:334] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.785s (SENT)
I20260812 19:08:03.393198 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 832) took 1322 ms. Trace:
W20260812 19:08:03.248735 2221 log.cc:924] Time spent T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.922s user 0.000s sys 0.001s
I20260812 19:08:03.301852 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2491) took 1047 ms. Trace:
I20260812 19:08:03.303166 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 796) took 1052 ms. Trace:
W20260812 19:08:03.307017 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 50574.1us
I20260812 19:08:03.394177 29450 rpcz_store.cc:276] 0812 19:08:02.250883 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.244013 (+993130us) service_pool.cc:227] Handling call
0812 19:08:03.258574 (+ 14561us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:03.301536 (+ 42962us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.303084 (+ 1548us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.394737 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 797) took 1094 ms. Trace:
I20260812 19:08:03.204492 2004 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.393843 29445 rpcz_store.cc:276] 0812 19:08:02.254517 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.862836 (+608319us) service_pool.cc:227] Handling call
0812 19:08:03.301782 (+438946us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.229070 2118 raft_consensus.cc:3032] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:03.256313 1362 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:03.396128 2118 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:03.397110 2004 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:03.327172 29451 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:03.393567 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2194) took 1123 ms. Trace:
I20260812 19:08:03.397838 29572 rpcz_store.cc:276] 0812 19:08:02.270634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.788579 (+517945us) service_pool.cc:227] Handling call
0812 19:08:03.393499 (+604920us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.393594 30625 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:03.302802 2232 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:03.401713 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 182988us
I20260812 19:08:03.402879 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 797) took 1151 ms. Trace:
W20260812 19:08:03.346570 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.022s user 0.048s sys 0.047s
I20260812 19:08:03.307329 2063 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:03.308566 2148 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:03.393632 29334 rpcz_store.cc:276] 0812 19:08:02.070821 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.070940 (+ 119us) service_pool.cc:227] Handling call
0812 19:08:02.086324 (+ 15384us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.274185 (+187861us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.392942 (+1118757us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54001801d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:03.393111 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":332206464}
I20260812 19:08:03.418531 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:03.362779 29335 raft_consensus.cc:603] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:03.398654 1820 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:03.420694 1820 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.421321 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 800) took 1169 ms. Trace:
I20260812 19:08:03.421787 29450 rpcz_store.cc:276] 0812 19:08:02.251863 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.398891 (+1147028us) service_pool.cc:227] Handling call
0812 19:08:03.420829 (+ 21938us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:03.421007 (+ 178us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:03.421032 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:03.421085 (+ 53us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:03.421115 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.421163 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:03.421263 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.422899 1820 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:03.423360 29450 raft_consensus.cc:1261] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:08:03.426600 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:03.427412 29576 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.109s user 0.017s sys 0.017s
I20260812 19:08:03.243319 29458 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:03.232200 2191 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:03.378403 2217 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:03.393332 2210 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:03.327649 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 803) took 1075 ms. Trace:
I20260812 19:08:03.309913 2092 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:03.249766 2100 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:03.393577 29452 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260812 19:08:03.331748 1967 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:08:03.403330 30625 leader_election.cc:334] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.858s (SENT)
I20260812 19:08:03.403262 29451 rpcz_store.cc:276] 0812 19:08:02.251118 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.326571 (+1075453us) service_pool.cc:227] Handling call
0812 19:08:03.326984 (+ 413us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:03.402589 (+ 75605us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.402808 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:03.403584 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.079s user 0.048s sys 0.047s
I20260812 19:08:03.395090 29574 rpcz_store.cc:276] 0812 19:08:02.300207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.799787 (+499580us) service_pool.cc:227] Handling call
0812 19:08:02.801643 (+ 1856us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:03.393921 (+592278us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:03.393965 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:03.394003 (+ 38us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:03.394038 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.394539 (+ 501us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b5400157054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:03.394578 (+ 39us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:03.394687 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1361601280}
I20260812 19:08:03.396715 1362 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.414528 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:03.301586 1986 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260812 19:08:03.429806 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 790) took 1182 ms. Trace:
I20260812 19:08:03.258477 2235 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.348351 2146 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:03.430946 29576 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.113s user 0.017s sys 0.017s
I20260812 19:08:03.434011 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 790) took 1208 ms. Trace:
I20260812 19:08:03.386121 29341 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:03.441243 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2155) took 1149 ms. Trace:
I20260812 19:08:03.307016 2056 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:03.392988 2244 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:03.333233 2003 consensus_queue.cc:1035] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260812 19:08:03.378077 2207 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:03.334127 1649 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260812 19:08:03.396957 2118 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.252534 30674 rpcz_store.cc:276] 0812 19:08:01.143314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:01.145442 (+ 2128us) service_pool.cc:227] Handling call
0812 19:08:01.278386 (+132944us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:01.908725 (+630339us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:02.648287 (+739562us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54004605d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:02.735002 (+ 86715us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1333112448}
I20260812 19:08:03.986758 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:04.001698 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.184s user 0.004s sys 0.047s
W20260812 19:08:04.219456 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.402s user 0.012s sys 0.050s
W20260812 19:08:03.475036 29332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2257) took 2103 ms (client timeout 2112 ms). Trace:
W20260812 19:08:03.482203 29579 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.210s user 0.013s sys 0.022s
W20260812 19:08:04.220942 29579 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.949s user 0.030s sys 0.048s
W20260812 19:08:04.220145 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.40346e+06us
W20260812 19:08:03.674090 30679 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.384s user 0.010s sys 0.039s
I20260812 19:08:04.186594 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:03.623792 29577 raft_consensus.cc:2379] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:04.114172 29456 rpcz_store.cc:276] 0812 19:08:02.252631 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.206776 (+954145us) service_pool.cc:227] Handling call
0812 19:08:03.327585 (+120809us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:04.114530 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 798) took 1863 ms. Trace:
I20260812 19:08:04.187088 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:04.187505 29458 rpcz_store.cc:276] 0812 19:08:02.248445 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.242761 (+994316us) service_pool.cc:227] Handling call
0812 19:08:03.243112 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:03.429537 (+186425us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.429745 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:04.188009 29569 rpcz_store.cc:276] 0812 19:08:02.225496 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.226578 (+ 1082us) service_pool.cc:227] Handling call
0812 19:08:02.236698 (+ 10120us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.366567 (+129869us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.366601 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:02.366643 (+ 42us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:02.366680 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.433756 (+1067076us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400037ad4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:03.433805 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:03.433935 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":298114816}
W20260812 19:08:04.220317 29332 rpcz_store.cc:269] 0812 19:08:01.326117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:01.326537 (+ 420us) service_pool.cc:227] Handling call
0812 19:08:03.429821 (+2103284us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54003d8454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:03.429932 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2122414848}
I20260812 19:08:04.264344 29452 rpcz_store.cc:276] 0812 19:08:02.251355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.393006 (+1141651us) service_pool.cc:227] Handling call
0812 19:08:03.393371 (+ 365us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:04.114238 (+720867us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.114468 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.628888 30684 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260812 19:08:04.187564 29450 rpcz_store.cc:276] 0812 19:08:02.291903 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.422707 (+1130804us) service_pool.cc:227] Handling call
0812 19:08:03.423020 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:03.423749 (+ 729us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:03.441176 (+ 17427us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:03.399401 2175 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:03.393204 2144 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:03.621810 29403 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.839s user 0.021s sys 0.047s
I20260812 19:08:04.188163 30675 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
W20260812 19:08:04.276063 29403 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.494s user 0.033s sys 0.065s
W20260812 19:08:03.992677 2236 log.cc:924] Time spent T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.593s user 0.001s sys 0.000s
I20260812 19:08:04.222488 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:04.283566 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 61172.2us
I20260812 19:08:04.284408 29399 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:03.605146 30670 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.757s user 0.044s sys 0.156s
W20260812 19:08:04.286157 30670 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.450s user 0.079s sys 0.240s
W20260812 19:08:04.147706 2238 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.818s user 0.001s sys 0.001s
I20260812 19:08:03.455736 2148 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:04.207273 1967 raft_consensus.cc:686] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:04.033541 2230 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.090380 2247 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.268335 29451 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:03.538115 2068 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:03.450222 2219 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 1.030s
I20260812 19:08:03.498769 2217 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:03.621791 2212 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:04.272509 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 869146us
I20260812 19:08:03.450354 2239 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:08:04.276587 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 1.92211e+06us
I20260812 19:08:04.278827 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:04.279601 2246 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.190s user 0.000s sys 0.000s
W20260812 19:08:04.279850 29400 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.043s user 0.002s sys 0.031s
W20260812 19:08:04.317395 29400 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.879s user 0.003s sys 0.032s
I20260812 19:08:04.214809 2244 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:03.417603 2004 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.806s
I20260812 19:08:04.281929 29458 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260812 19:08:04.285611 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:04.290670 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.046s user 0.030s sys 0.048s
I20260812 19:08:04.323493 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2157) took 2031 ms. Trace:
I20260812 19:08:04.220144 30681 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
W20260812 19:08:04.267750 2218 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.880s user 0.011s sys 0.015s
I20260812 19:08:04.323886 29458 rpcz_store.cc:276] 0812 19:08:02.292528 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.277212 (+1984684us) service_pool.cc:227] Handling call
0812 19:08:04.277531 (+ 319us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:04.323240 (+ 45709us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.323435 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:04.320356 29403 connection.cc:232] RPC call timeout handler was delayed by -1.5635673531796783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260812 19:08:04.295855 29446 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.934s user 0.000s sys 0.012s
I20260812 19:08:03.439337 2063 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:03.417841 2232 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:04.271780 30679 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.973s user 0.010s sys 0.072s
W20260812 19:08:04.297432 29528 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.474s user 0.000s sys 0.014s
I20260812 19:08:04.223198 2263 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.296984 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2158) took 2004 ms. Trace:
I20260812 19:08:04.304028 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2156) took 2011 ms. Trace:
I20260812 19:08:03.450605 2081 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.227481 1741 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:04.270843 2100 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:03.703326 2194 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.569s
W20260812 19:08:04.312285 30625 leader_election.cc:334] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.890s (SENT)
I20260812 19:08:04.340597 29567 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:04.204041 2092 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.292205 2118 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:04.307749 29445 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.910s user 0.003s sys 0.013s
I20260812 19:08:03.433231 2117 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:04.318876 29282 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.815s user 0.016s sys 0.039s
W20260812 19:08:04.365360 29282 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.862s user 0.016s sys 0.039s
I20260812 19:08:04.313670 2247 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.282490 2213 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:04.294919 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:04.323591 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.079s user 0.030s sys 0.048s
W20260812 19:08:04.328465 29446 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.966s user 0.001s sys 0.012s
W20260812 19:08:04.306674 2209 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.911s user 0.013s sys 0.014s
I20260812 19:08:04.332540 29451 rpcz_store.cc:276] 0812 19:08:02.292235 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.205777 (+1913542us) service_pool.cc:227] Handling call
0812 19:08:04.268153 (+ 62376us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:04.303753 (+ 35600us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.303969 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:04.332037 29456 rpcz_store.cc:276] 0812 19:08:02.292811 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.277145 (+1984334us) service_pool.cc:227] Handling call
0812 19:08:04.296521 (+ 19376us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:04.296684 (+ 163us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:04.296709 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:04.296751 (+ 42us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:04.296785 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.296833 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.296930 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:04.335031 29528 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.511s user 0.000s sys 0.014s
W20260812 19:08:04.324644 1638 consensus_meta.cc:287] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 2.356s user 0.011s sys 0.002s
I20260812 19:08:04.306484 2230 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.311105 2068 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:04.328089 2218 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.940s user 0.011s sys 0.015s
I20260812 19:08:04.326490 2244 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.344736 2130 raft_consensus.cc:3032] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:04.382768 2130 raft_consensus.cc:729] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:04.383549 2172 raft_consensus.cc:3032] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:04.383731 2130 consensus_queue.cc:260] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:04.344426 2100 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.386502 2100 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.387068 2100 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:04.387391 1478 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:04.345078 2252 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.387802 1478 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:04.388082 2252 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.330278 2191 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:04.345487 2260 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:04.340490 2081 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.312744 2207 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:04.394126 29284 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.808s user 0.000s sys 0.015s
W20260812 19:08:04.394493 29284 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.889s user 0.000s sys 0.015s
I20260812 19:08:04.398177 2279 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.398802 2279 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.401238 2279 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.346030 2063 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.364024 29445 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s user 0.003s sys 0.017s
W20260812 19:08:04.338672 1717 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s user 0.013s sys 0.020s
W20260812 19:08:04.407142 1717 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s user 0.013s sys 0.020s
I20260812 19:08:04.366711 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2264) took 2309 ms. Trace:
I20260812 19:08:04.407256 2068 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:04.408422 2230 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.407599 29580 rpcz_store.cc:276] 0812 19:08:02.056868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.058295 (+ 1427us) service_pool.cc:227] Handling call
0812 19:08:02.085971 (+ 27676us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:02.092540 (+ 6569us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.092573 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:02.109865 (+ 17292us) log.cc:842] Serialized 26 byte log entry
0812 19:08:02.337498 (+227633us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:02.337544 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:02.337621 (+ 77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.351956 (+2014335us) raft_consensus.cc:1671] finished
0812 19:08:04.366476 (+ 14520us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.366645 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14578,"raft.queue_time_us":183046,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":29802,"thread_start_us":108858,"threads_started":1,"wal-append.queue_time_us":1091861}
I20260812 19:08:04.371446 2219 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.410275 29580 raft_consensus.cc:1226] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
W20260812 19:08:04.370697 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.12578e+06us
W20260812 19:08:04.372470 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 1.5682e+06us
I20260812 19:08:04.411656 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 789) took 2242 ms. Trace:
I20260812 19:08:04.412087 29579 rpcz_store.cc:276] 0812 19:08:02.169426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.169584 (+ 158us) service_pool.cc:227] Handling call
0812 19:08:02.234206 (+ 64622us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:02.234511 (+ 305us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.234543 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:02.235022 (+ 479us) log.cc:842] Serialized 26 byte log entry
0812 19:08:04.263497 (+2028475us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:04.263546 (+ 49us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.263617 (+ 71us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.411434 (+147817us) raft_consensus.cc:1671] finished
0812 19:08:04.411457 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.411593 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":70082,"raft.queue_time_us":89828,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":27518,"thread_start_us":1949155,"threads_started":1,"wal-append.queue_time_us":2065359}
I20260812 19:08:04.414252 2146 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.340612 2263 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.416335 29574 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:08:04.416884 2176 consensus_meta.cc:287] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.077s user 0.008s sys 0.002s
I20260812 19:08:04.417253 2263 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.393811 2229 log.cc:924] Time spent T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 2.075s user 0.014s sys 0.007s
I20260812 19:08:04.323544 2212 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.417426 2176 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.418941 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 828) took 2391 ms. Trace:
I20260812 19:08:04.419407 2212 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.420688 2062 consensus_meta.cc:287] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.538s user 0.005s sys 0.006s
I20260812 19:08:04.375038 2004 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.348198 1967 consensus_queue.cc:237] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.419296 29339 rpcz_store.cc:276] 0812 19:08:02.028411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.028595 (+ 184us) service_pool.cc:227] Handling call
0812 19:08:02.028917 (+ 322us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:02.038729 (+ 9812us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.038762 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:02.039268 (+ 506us) log.cc:842] Serialized 26 byte log entry
0812 19:08:02.160761 (+121493us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:02.160802 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:02.160870 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.393207 (+2232337us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:08:04.418660 (+ 25453us) raft_consensus.cc:1671] finished
0812 19:08:04.418679 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.418881 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14361,"raft.queue_time_us":10822,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3860,"spinlock_wait_cycles":27474304,"thread_start_us":121181,"threads_started":1,"wal-append.queue_time_us":225332}
W20260812 19:08:04.346457 30625 leader_election.cc:334] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.897s (SENT)
I20260812 19:08:04.345944 2190 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:04.342458 2251 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260812 19:08:04.371708 2209 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s user 0.013s sys 0.014s
I20260812 19:08:04.347680 2092 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.402475 2172 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:04.409989 2219 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.375138 2262 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.368587 29569 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:04.382503 2056 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:04.340797 2239 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.413803 2148 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260812 19:08:04.422410 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:04.422773 2191 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.424202 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:04.425002 29528 leader_election.cc:334] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.307s (SENT)
I20260812 19:08:04.425235 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:04.512540 29528 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:04.345686 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 896) took 2369 ms. Trace:
I20260812 19:08:04.347288 2232 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.427291 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2270) took 1015 ms. Trace:
I20260812 19:08:04.428086 29579 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:04.431275 30625 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:04.319849 29283 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.789s user 0.022s sys 0.026s
I20260812 19:08:04.376262 29326 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:05.338582 2047 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:05.339143 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 838) took 2146 ms. Trace:
I20260812 19:08:05.339566 29326 rpcz_store.cc:276] 0812 19:08:03.192325 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.195840 (+ 3515us) service_pool.cc:227] Handling call
0812 19:08:05.338953 (+2143113us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002319d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:05.339082 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2583616768}
I20260812 19:08:04.675058 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:04.681617 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:37750 (request call id 1008) took 1270 ms. Trace:
I20260812 19:08:04.620064 2284 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:04.753242 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:05.341042 30629 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:05.341368 2284 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:04.757170 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 378480us
I20260812 19:08:05.343706 2216 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:05.340730 29337 rpcz_store.cc:276] 0812 19:08:03.411236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.411460 (+ 224us) service_pool.cc:227] Handling call
0812 19:08:04.681020 (+1269560us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b54003903d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6ed53c6e 00007f9c68456807 00007f9c68456792 00007f9c684566c1 00007f9c6845663c 00007f9c684558f2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:04.681547 (+ 527us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1785781120}
I20260812 19:08:05.339186 2047 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.774196 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:04.761691 29281 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.567s user 0.000s sys 0.024s
W20260812 19:08:05.346329 29281 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.152s user 0.002s sys 0.024s
I20260812 19:08:05.344134 2216 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:05.346851 2216 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:05.348163 2283 consensus_queue.cc:1046] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:04.793445 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 795) took 2542 ms. Trace:
I20260812 19:08:05.349181 29446 rpcz_store.cc:276] 0812 19:08:02.250646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.231485 (+980839us) service_pool.cc:227] Handling call
0812 19:08:03.231844 (+ 359us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:08:03.232166 (+ 322us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:03.232195 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:08:03.251692 (+ 19497us) log.cc:842] Serialized 52 byte log entry
0812 19:08:04.372682 (+1120990us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:04.777630 (+404948us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.792917 (+ 15287us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.793121 (+ 204us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.793205 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.793242 (+ 37us) raft_consensus.cc:1671] finished
0812 19:08:04.793258 (+ 16us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.793386 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":14844,"prepare.queue_time_us":435523,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":375720,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":366651,"thread_start_us":1385204,"threads_started":2,"wal-append.queue_time_us":1172393}
I20260812 19:08:05.351393 29446 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:05.351900 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 897) took 3375 ms. Trace:
I20260812 19:08:05.352255 30670 rpcz_store.cc:276] 0812 19:08:01.976215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:01.976328 (+ 113us) service_pool.cc:227] Handling call
0812 19:08:01.976658 (+ 330us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:01.983637 (+ 6979us) raft_consensus.cc:1520] Early marking committed up to index 9
0812 19:08:02.174379 (+190742us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.468187 (+2293808us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.469125 (+ 938us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:04.469608 (+ 483us) log.cc:842] Serialized 26 byte log entry
0812 19:08:05.351371 (+881763us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540050d0a0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69d9e028 00007f9c69e0ea7b 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c
0812 19:08:05.351433 (+ 62us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:05.351481 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:05.351565 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:05.351695 (+ 130us) raft_consensus.cc:1671] finished
0812 19:08:05.351710 (+ 15us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:05.351831 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":245807,"prepare.queue_time_us":715285,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":246387,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":71833,"spinlock_wait_cycles":551498112,"thread_start_us":2161526,"threads_started":4,"wal-append.queue_time_us":986586}
W20260812 19:08:04.830480 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 62865us
I20260812 19:08:04.850265 29334 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260812 19:08:04.660540 29567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2272) took 1248 ms (client timeout 1610 ms). Trace:
W20260812 19:08:05.386445 29567 rpcz_store.cc:269] 0812 19:08:03.412260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.418259 (+ 5999us) service_pool.cc:227] Handling call
0812 19:08:04.660339 (+1242080us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400178754. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:04.660453 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2119011200}
I20260812 19:08:05.387796 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:04.661206 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2274) took 1213 ms. Trace:
W20260812 19:08:04.658249 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2195) took 2387 ms (client timeout 1540 ms). Trace:
I20260812 19:08:05.400583 29569 rpcz_store.cc:276] 0812 19:08:03.447656 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.263338 (+815682us) service_pool.cc:227] Handling call
0812 19:08:04.661049 (+397711us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5400156654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:04.661152 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":76816256}
W20260812 19:08:05.401203 29577 rpcz_store.cc:269] 0812 19:08:02.271180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.318406 (+ 47226us) service_pool.cc:227] Handling call
0812 19:08:04.658002 (+2339596us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400172354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:04.658163 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2886147328}
I20260812 19:08:04.714375 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2271) took 1302 ms. Trace:
I20260812 19:08:05.411471 29579 rpcz_store.cc:276] 0812 19:08:03.411991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.413526 (+1001535us) service_pool.cc:227] Handling call
0812 19:08:04.413842 (+ 316us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:04.714075 (+300233us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.714313 (+ 238us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.412747 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:05.413393 29579 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:08:04.713677 29580 rpcz_store.cc:276] 0812 19:08:03.411700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.409588 (+997888us) service_pool.cc:227] Handling call
0812 19:08:04.410056 (+ 468us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:04.427043 (+ 16987us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.427236 (+ 193us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:04.613296 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 188318us
I20260812 19:08:04.793329 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 788) took 2631 ms. Trace:
I20260812 19:08:04.467434 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2192) took 2199 ms. Trace:
I20260812 19:08:04.870231 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 799) took 2618 ms. Trace:
I20260812 19:08:04.490887 30623 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:05.421296 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 930463us
W20260812 19:08:04.725111 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 378472us
I20260812 19:08:05.426195 2211 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:05.426679 2211 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:05.427000 2211 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:04.425676 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.432227 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 944) took 1009 ms. Trace:
I20260812 19:08:05.432586 30674 rpcz_store.cc:276] 0812 19:08:04.423176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.425390 (+ 2214us) service_pool.cc:227] Handling call
0812 19:08:05.432152 (+1006762us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:04.515326 30679 rpcz_store.cc:276] 0812 19:08:01.975915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:01.982438 (+ 6523us) service_pool.cc:227] Handling call
0812 19:08:02.017444 (+ 35006us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.025451 (+ 8007us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:08:02.284741 (+259290us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.345317 (+2060576us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:04.345372 (+ 55us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:04.345431 (+ 59us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.345503 (+ 72us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.345631 (+ 128us) inbound_call.cc:173] Queueing success response
0812 19:08:04.385008 (+ 39377us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b180033b440. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ebff96 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ef2638 00007f9c66ef5b11
0812 19:08:04.386342 (+ 1334us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180033b440. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ef2638 00007f9c66ef5b11 00007f9c66ef5ac9 00007f9c66ef5a59 00007f9c66ef5a21 00007f9c66ef4d1d 00007f9c7014a204 00007f9c7014a039 00007f9c66ed4dc6
Metrics: {"mutex_wait_us":1602,"prepare.queue_time_us":309507,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5854,"raft.queue_time_us":5051,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14818,"spinlock_wait_cycles":44493440,"thread_start_us":2256683,"threads_started":2,"wal-append.queue_time_us":2057712}
I20260812 19:08:05.435173 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.435688 2331 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:05.436492 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.436337 2331 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:05.437327 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 950) took 1012 ms. Trace:
I20260812 19:08:04.512794 30686 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:05.439409 30686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 948) took 1014 ms. Trace:
I20260812 19:08:05.439719 30686 rpcz_store.cc:276] 0812 19:08:04.424818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.424981 (+ 163us) service_pool.cc:227] Handling call
0812 19:08:05.439351 (+1014370us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.440562 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:04.535998 1478 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:04.623355 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.445338 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 951) took 1019 ms. Trace:
I20260812 19:08:05.445662 30685 rpcz_store.cc:276] 0812 19:08:04.425667 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.426914 (+ 1247us) service_pool.cc:227] Handling call
0812 19:08:05.445280 (+1018366us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.446493 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.447055 30685 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:05.447072 2331 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:05.447855 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:05.449075 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:04.623373 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.450609 30681 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:05.451056 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 945) took 1027 ms. Trace:
I20260812 19:08:05.451333 30681 rpcz_store.cc:276] 0812 19:08:04.423486 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.426579 (+ 3093us) service_pool.cc:227] Handling call
0812 19:08:05.451001 (+1024422us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":42112}
I20260812 19:08:05.452337 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:05.452857 30681 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:04.687808 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 908) took 1274 ms. Trace:
I20260812 19:08:05.454862 30676 rpcz_store.cc:276] 0812 19:08:03.413608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.413760 (+ 152us) service_pool.cc:227] Handling call
0812 19:08:03.450968 (+ 37208us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:03.607988 (+157020us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:03.608028 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:04.190278 (+582250us) log.cc:842] Serialized 26 byte log entry
0812 19:08:04.597710 (+407432us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:04.597763 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.597850 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.687536 (+ 89686us) raft_consensus.cc:1671] finished
0812 19:08:04.687560 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.687739 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":146624,"raft.queue_time_us":265983,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2538,"thread_start_us":325372,"threads_started":1,"wal-append.queue_time_us":431262}
I20260812 19:08:05.456584 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:04.751607 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 910) took 1310 ms. Trace:
W20260812 19:08:04.604142 1807 consensus_meta.cc:287] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 2.301s user 0.012s sys 0.000s
I20260812 19:08:04.665118 1839 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:05.419142 29445 rpcz_store.cc:276] 0812 19:08:02.251612 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.396148 (+1144536us) service_pool.cc:227] Handling call
0812 19:08:03.396472 (+ 324us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:03.396684 (+ 212us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:03.396707 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:03.397147 (+ 440us) log.cc:842] Serialized 26 byte log entry
0812 19:08:04.406036 (+1008889us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:04.868758 (+462722us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.869888 (+ 1130us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.869986 (+ 98us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.870033 (+ 47us) raft_consensus.cc:1671] finished
0812 19:08:04.870044 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.870151 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":386230,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":109,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":1085022,"threads_started":2,"wal-append.queue_time_us":1058453}
I20260812 19:08:05.463726 29445 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:05.464845 29445 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:04.667001 1945 raft_consensus.cc:3032] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:05.466895 1945 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:04.794871 1967 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:05.468947 30623 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:05.469861 2132 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:05.469626 1967 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.470362 2132 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:05.470724 2132 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:04.773435 1989 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:05.474947 1989 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.477217 1989 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.506605 2004 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.480131 2004 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.379252 2001 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.554979 2056 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:04.437058 2062 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.437698 30679 rpcz_store.cc:276] 0812 19:08:04.425409 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.436263 (+1010854us) service_pool.cc:227] Handling call
0812 19:08:05.437273 (+ 1010us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.488052 2062 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.637553 2042 consensus_meta.cc:287] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.298s user 0.010s sys 0.002s
I20260812 19:08:05.492317 1967 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.627633 2073 consensus_meta.cc:287] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 2.310s user 0.005s sys 0.006s
I20260812 19:08:04.535074 2081 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.555249 2092 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:04.682180 2102 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:04.725786 2118 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:05.509402 2102 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.510454 30686 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:05.511747 2102 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.642988 2130 raft_consensus.cc:3037] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:04.934767 2125 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:05.515246 2125 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:05.522095 29177 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55514 (request call id 388) took 12317 ms (client timeout 15000 ms). Trace:
I20260812 19:08:04.608513 2172 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.612692 2176 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.535481 2191 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.532634 2191 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.613494 2195 raft_consensus.cc:3032] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:05.535593 2195 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:04.764628 2207 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:05.538072 29179 catalog_manager.cc:5495] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:05.540524 29179 catalog_manager.cc:5495] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:08:04.446569 2212 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:05.545130 2212 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:04.414265 2214 consensus_meta.cc:287] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.119s user 0.010s sys 0.001s
I20260812 19:08:05.547111 2214 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:04.424187 2215 consensus_meta.cc:287] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.076s user 0.006s sys 0.003s
I20260812 19:08:05.552709 2215 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:04.691638 2217 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.555823 2217 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.623073 2219 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.402441 2260 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:05.558053 2260 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.559484 2172 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:05.559978 2260 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:04.603531 2258 log.cc:924] Time spent T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.217s user 0.001s sys 0.002s
W20260812 19:08:04.652275 2222 log.cc:924] Time spent T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 2.333s user 0.013s sys 0.023s
W20260812 19:08:04.612629 2271 log.cc:924] Time spent T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.211s user 0.000s sys 0.001s
I20260812 19:08:04.507083 2262 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:05.569023 2262 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:04.505272 2275 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:05.570376 2275 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:05.573460 29456 raft_consensus.cc:1261] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:04.611215 2244 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.611819 2247 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.771178 2235 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:04.514035 2239 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:08:05.153324 2252 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:05.218988 29452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2159) took 2915 ms. Trace:
I20260812 19:08:05.264039 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.266345 1722 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:05.281101 29574 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2273) took 1868 ms (client timeout 1973 ms). Trace:
I20260812 19:08:05.586239 29451 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:05.285270 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41158 (request call id 26) took 3184 ms. Trace:
I20260812 19:08:05.586489 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 949) took 1161 ms. Trace:
I20260812 19:08:05.317732 2322 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:05.320972 29283 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.816s user 0.023s sys 0.031s
I20260812 19:08:05.326802 2264 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:05.344658 2122 consensus_queue.cc:1035] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:04.775355 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.355640 1810 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:05.356379 2248 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:05.346467 2199 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
W20260812 19:08:05.346879 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.023s user 0.007s sys 0.008s
W20260812 19:08:05.598562 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.275s user 0.008s sys 0.008s
W20260812 19:08:05.599128 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.31351e+06us
I20260812 19:08:05.599407 1810 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.343812 2284 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:05.358196 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:05.603022 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:05.604221 2248 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.359305 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.359617 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.360363 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.592566 2322 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:05.606735 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.361044 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.360877 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.607697 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 953) took 1181 ms. Trace:
I20260812 19:08:05.361724 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:05.362210 2253 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:05.364667 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:05.367029 29446 raft_consensus.cc:1226] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 2. Current term is 3. Ops: []
I20260812 19:08:05.379098 29326 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:05.385047 2216 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.389339 2123 raft_consensus.cc:3032] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:05.390105 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:05.402997 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:05.404337 2047 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:05.404999 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:05.415620 29580 raft_consensus.cc:1226] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:05.419031 29576 rpcz_store.cc:276] 0812 19:08:02.267819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.267950 (+ 131us) service_pool.cc:227] Handling call
0812 19:08:02.287752 (+ 19802us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.287948 (+ 196us) raft_consensus.cc:1520] Early marking committed up to index 5
0812 19:08:04.186904 (+1898956us) log.cc:842] Serialized 14 byte log entry
0812 19:08:04.466995 (+280091us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:04.467066 (+ 71us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:04.467131 (+ 65us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.467209 (+ 78us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.467359 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":36015,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":111,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":2148258,"threads_started":2,"wal-append.queue_time_us":478925}
I20260812 19:08:05.442116 2333 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:05.443719 2220 consensus_queue.cc:1046] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260812 19:08:05.449586 29528 connection.cc:232] RPC call timeout handler was delayed by -1.1617222409695387s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2939s for a response
I20260812 19:08:05.449175 2334 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:05.457765 30671 rpcz_store.cc:276] 0812 19:08:03.440815 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.440965 (+ 150us) service_pool.cc:227] Handling call
0812 19:08:04.113825 (+672860us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:04.206665 (+ 92840us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:04.206698 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:04.207176 (+ 478us) log.cc:842] Serialized 26 byte log entry
0812 19:08:04.656860 (+449684us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180033b860. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ed4345 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb 00007f9c69a95063 00007f9c69a95411 00007f9c69de25cf
0812 19:08:04.743244 (+ 86384us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:04.743272 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.743326 (+ 54us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:04.751289 (+ 7963us) raft_consensus.cc:1671] finished
0812 19:08:04.751312 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.751543 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":93283,"raft.queue_time_us":111546,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7348,"spinlock_wait_cycles":6247296,"thread_start_us":86165,"threads_started":1,"wal-append.queue_time_us":522700}
I20260812 19:08:05.459926 1807 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.467736 1986 consensus_queue.cc:1046] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:05.467993 2327 consensus_queue.cc:1046] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 19:08:05.474211 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 52320.5us
I20260812 19:08:05.418432 29578 rpcz_store.cc:276] 0812 19:08:02.161770 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.161906 (+ 136us) service_pool.cc:227] Handling call
0812 19:08:02.162236 (+ 330us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:02.773618 (+611382us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:02.773653 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:02.773694 (+ 41us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:02.773731 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:04.793061 (+2019330us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b540012b454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:04.793116 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:04.793261 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1405581184}
W20260812 19:08:05.482856 30625 connection.cc:232] RPC call timeout handler was delayed by -3.1213134319987148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
I20260812 19:08:05.465909 29445 raft_consensus.cc:1226] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:05.433698 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:05.481364 2001 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:05.488495 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:05.484014 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 952) took 1058 ms. Trace:
I20260812 19:08:05.491230 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.492204 2042 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:04.666224 2071 raft_consensus.cc:3032] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:08:05.496466 2211 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.497941 30681 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:05.467859 1945 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.498299 2132 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.502408 30675 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:05.506871 2073 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.508683 2092 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:04.387617 2100 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:04.621529 2145 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:05.517369 2125 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:05.550299 29572 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:05.553354 30676 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:05.560734 30686 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:05.583258 29456 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:05.542328 29179 catalog_manager.cc:5495] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 3 to 5, now has a pending config: VOTER 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65), VOTER 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66), VOTER dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67), VOTER a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 5 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:05.586931 30684 rpcz_store.cc:276] 0812 19:08:04.425124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.536224 (+111100us) service_pool.cc:227] Handling call
0812 19:08:05.263851 (+727627us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:05.586429 (+322578us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":10167552}
W20260812 19:08:05.586365 29574 rpcz_store.cc:269] 0812 19:08:03.412548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.186808 (+774260us) service_pool.cc:227] Handling call
0812 19:08:05.280867 (+1094059us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400172854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:05.281030 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":350349184}
I20260812 19:08:05.585433 29452 rpcz_store.cc:276] 0812 19:08:02.303797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.265158 (+1961361us) service_pool.cc:227] Handling call
0812 19:08:04.280055 (+ 14897us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:04.663472 (+383417us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:04.663507 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:04.664019 (+ 512us) log.cc:842] Serialized 26 byte log entry
0812 19:08:04.728811 (+ 64792us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:04.728847 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:05.218506 (+489659us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b54000f91d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:05.218562 (+ 56us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:05.218625 (+ 63us) raft_consensus.cc:1671] finished
0812 19:08:05.218657 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:05.218923 (+ 266us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":12140,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":227421952,"thread_start_us":348434,"threads_started":2,"wal-append.queue_time_us":85112}
W20260812 19:08:05.561830 29177 rpcz_store.cc:269] 0812 19:07:53.204672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:53.216924 (+ 12252us) service_pool.cc:227] Handling call
0812 19:08:05.522023 (+12305099us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:08:01.734863 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:08:01.735395 (+ 532us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:08:01.735433 (+ 38us) write_op.cc:435] Acquired schema lock
0812 19:08:01.735444 (+ 11us) tablet.cc:660] Decoding operations
0812 19:08:01.736449 (+ 1005us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:08:01.736514 (+ 65us) write_op.cc:641] Partition lock acquired for write op
0812 19:08:01.736546 (+ 32us) tablet.cc:683] Acquiring locks for 11 operations
0812 19:08:01.850015 (+113469us) tablet.cc:699] Row locks acquired
0812 19:08:01.850032 (+ 17us) write_op.cc:260] PREPARE: finished
0812 19:08:01.850189 (+ 157us) write_op.cc:270] Start()
0812 19:08:01.850314 (+ 125us) write_op.cc:276] Timestamp: P: 1786561681850139 usec, L: 0
0812 19:08:01.850333 (+ 19us) op_driver.cc:331] REPLICATION: starting
0812 19:08:01.851107 (+ 774us) log.cc:842] Serialized 4387 byte log entry
0812 19:08:04.371276 (+2520169us) op_driver.cc:444] REPLICATION: finished
0812 19:08:04.630824 (+259548us) write_op.cc:301] APPLY: starting
0812 19:08:04.630935 (+ 111us) tablet.cc:1365] starting BulkCheckPresence
0812 19:08:04.631198 (+ 263us) tablet.cc:1368] finished BulkCheckPresence
0812 19:08:04.631331 (+ 133us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:08:04.632824 (+ 1493us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:08:04.633075 (+ 251us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0812 19:08:04.633121 (+ 46us) write_op.cc:312] APPLY: finished
0812 19:08:04.633478 (+ 357us) log.cc:842] Serialized 83 byte log entry
0812 19:08:04.635551 (+ 2073us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:04.635710 (+ 159us) write_op.cc:454] Released schema lock
0812 19:08:04.635759 (+ 49us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":227105,"apply.run_cpu_time_us":3418,"apply.run_wall_time_us":5401,"num_ops":11,"prepare.queue_time_us":455230,"prepare.run_cpu_time_us":16985,"prepare.run_wall_time_us":358810,"raft.queue_time_us":941798,"raft.run_cpu_time_us":21150,"raft.run_wall_time_us":381569,"replication_time_us":2492603,"thread_start_us":1746712,"threads_started":4,"wal-append.queue_time_us":437121}]]}
I20260812 19:08:05.586887 29335 rpcz_store.cc:276] 0812 19:08:02.100866 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.100982 (+ 116us) service_pool.cc:227] Handling call
0812 19:08:05.285030 (+3184048us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400218754. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e08bc3 00007f9c6ed5281a 00007f9c68451d04 00007f9c68451c92 00007f9c68451bc1 00007f9c68451b3c 00007f9c68450df2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:05.285176 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2471762304}
I20260812 19:08:05.590153 29567 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:05.549131 2214 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:05.594509 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.599807 29399 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:05.351327 29453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 787) took 3104 ms (client timeout 1770 ms). Trace:
I20260812 19:08:05.606804 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.363504 1362 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:05.863227 29446 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2170) took 1186 ms. Trace:
W20260812 19:08:06.127095 30625 connection.cc:232] RPC call timeout handler was delayed by -3.0825170718599111s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260812 19:08:06.127688 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 763045us
I20260812 19:08:06.128481 29332 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:06.128155 30685 rpcz_store.cc:276] 0812 19:08:04.425937 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.448841 (+1022904us) service_pool.cc:227] Handling call
0812 19:08:05.483949 (+ 35108us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:06.129724 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 529962us
I20260812 19:08:05.608069 30670 rpcz_store.cc:276] 0812 19:08:04.426247 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.360796 (+934549us) service_pool.cc:227] Handling call
0812 19:08:05.607631 (+246835us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.662658 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.131196 29399 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.131297 29569 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:05.696792 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:06.132436 29453 rpcz_store.cc:269] 0812 19:08:02.246707 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.299634 (+ 52927us) service_pool.cc:227] Handling call
0812 19:08:05.351082 (+3051448us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b5400310954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:05.351223 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6709149056}
W20260812 19:08:06.126722 29528 connection.cc:232] RPC call timeout handler was delayed by -1.5992492490913719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
W20260812 19:08:06.132953 29569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 799) took 1515 ms (client timeout 1835 ms). Trace:
I20260812 19:08:05.608289 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:06.133796 29528 connection.cc:232] RPC call timeout handler was delayed by -1.587549249175936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260812 19:08:06.124359 29446 rpcz_store.cc:276] 0812 19:08:04.676819 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.353829 (+677010us) service_pool.cc:227] Handling call
0812 19:08:05.366828 (+ 12999us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:05.862854 (+496026us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:05.863153 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260812 19:08:05.881036 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 320995us
I20260812 19:08:06.123795 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 798) took 1506 ms. Trace:
I20260812 19:08:06.132957 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.129007 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2174) took 1451 ms. Trace:
I20260812 19:08:06.127910 30623 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:06.133884 29569 rpcz_store.cc:269] 0812 19:08:04.617608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.402681 (+785073us) service_pool.cc:227] Handling call
0812 19:08:06.132891 (+730210us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.136106 29445 rpcz_store.cc:276] 0812 19:08:04.677800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.465461 (+787661us) service_pool.cc:227] Handling call
0812 19:08:05.465739 (+ 278us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.128756 (+663017us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.128952 (+ 196us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.126910 29577 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:06.137385 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 802) took 1412 ms. Trace:
I20260812 19:08:06.137807 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.135092 29574 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:08:06.138586 29337 raft_consensus.cc:2379] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:06.124521 29342 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:08:06.135900 29580 rpcz_store.cc:276] 0812 19:08:04.617326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.415071 (+797745us) service_pool.cc:227] Handling call
0812 19:08:05.415453 (+ 382us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.123447 (+707994us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.123723 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.137648 29577 rpcz_store.cc:276] 0812 19:08:04.724726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.404755 (+680029us) service_pool.cc:227] Handling call
0812 19:08:06.137333 (+732578us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.131747 29578 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 805) took 1406 ms. Trace:
I20260812 19:08:06.129756 29338 raft_consensus.cc:2362] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3. Re-sending same reply.
I20260812 19:08:06.140939 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:06.141184 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.141809 29331 raft_consensus.cc:2379] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.142061 29338 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:05.813082 29334 raft_consensus.cc:2454] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:06.142946 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:05.624148 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 800) took 1006 ms. Trace:
I20260812 19:08:06.129484 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.139786 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.143671 29579 rpcz_store.cc:276] 0812 19:08:04.617850 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.481015 (+863165us) service_pool.cc:227] Handling call
0812 19:08:05.481342 (+ 327us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:05.481542 (+ 200us) raft_consensus.cc:1520] Early marking committed up to index 9
0812 19:08:05.587820 (+106278us) log.cc:842] Serialized 14 byte log entry
0812 19:08:05.623826 (+ 36006us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:05.623876 (+ 50us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:05.623920 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:05.623987 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:05.624094 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":608438,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":22582,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":34161,"thread_start_us":141607,"threads_started":2,"wal-append.queue_time_us":387092}
I20260812 19:08:06.145227 29342 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
I20260812 19:08:06.145551 29337 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:08:06.145588 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.127180 29341 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:08:06.138675 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 806) took 1413 ms. Trace:
I20260812 19:08:06.144055 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.146777 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.145746 29578 rpcz_store.cc:276] 0812 19:08:04.725380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.130958 (+1405578us) service_pool.cc:227] Handling call
0812 19:08:06.131268 (+ 310us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.131456 (+ 188us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:06.131482 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:06.131517 (+ 35us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:06.131546 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.131593 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:06.131695 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.148401 29337 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:08:06.147173 29334 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:06.148532 29574 rpcz_store.cc:276] 0812 19:08:04.725572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.134445 (+1408873us) service_pool.cc:227] Handling call
0812 19:08:06.134738 (+ 293us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.138439 (+ 3701us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.138623 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.151401 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.153438 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.154301 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.155198 30681 raft_consensus.cc:2396] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260812 19:08:06.156664 29528 connection.cc:232] RPC call timeout handler was delayed by -1.419133777031675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29860000000000003s for a response
W20260812 19:08:06.157372 29528 leader_election.cc:334] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.939s (SENT)
I20260812 19:08:06.157862 29528 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.158566 29337 raft_consensus.cc:1226] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:06.052680 2004 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.159373 2214 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:06.159765 2004 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:06.161379 30625 connection.cc:232] RPC call timeout handler was delayed by -3.0592463188804686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260812 19:08:05.992105 2212 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:06.053042 2339 log.cc:924] Time spent T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260812 19:08:06.151283 2122 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:06.145987 2071 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:06.150993 2125 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.161928 2004 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.166460 29457 raft_consensus.cc:2379] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.167331 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2268) took 2761 ms. Trace:
W20260812 19:08:06.161991 30625 connection.cc:232] RPC call timeout handler was delayed by -2.1433827299624681s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
I20260812 19:08:06.168020 29341 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260812 19:08:06.172281 30625 connection.cc:232] RPC call timeout handler was delayed by -1.9809684618376195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1638s for a response
I20260812 19:08:06.126803 29179 catalog_manager.cc:5495] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 4 to 7, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 7 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:08:06.172640 30625 connection.cc:232] RPC call timeout handler was delayed by -1.9431684617884457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
I20260812 19:08:06.170393 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:06.173738 30625 connection.cc:232] RPC call timeout handler was delayed by -1.8369684619829059s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260812 19:08:06.174486 29338 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:06.174444 30625 connection.cc:232] RPC call timeout handler was delayed by -1.5366641809232533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
I20260812 19:08:06.171723 29572 rpcz_store.cc:276] 0812 19:08:03.406339 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:03.406833 (+ 494us) service_pool.cc:227] Handling call
0812 19:08:03.407164 (+ 331us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.130258 (+2723094us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.167079 (+ 36821us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54000ae954. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:06.167274 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4928451456}
I20260812 19:08:06.176995 29179 catalog_manager.cc:5495] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 2 to 3, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:08:06.180315 30675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 946) took 1756 ms (client timeout 2045 ms). Trace:
I20260812 19:08:05.624961 2284 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.180958 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:06.180819 30675 rpcz_store.cc:269] 0812 19:08:04.423802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.423929 (+ 127us) service_pool.cc:227] Handling call
0812 19:08:06.180098 (+1756169us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b540070e9d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:06.180244 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2276757760}
W20260812 19:08:06.182603 29567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 803) took 1457 ms (client timeout 1561 ms). Trace:
I20260812 19:08:06.162091 2100 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.189605 1945 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:06.191102 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2163) took 1516 ms. Trace:
I20260812 19:08:06.191206 2322 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.162429 2262 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.192157 29575 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:06.164774 2122 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.193226 2122 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:05.890259 2248 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:05.702040 1810 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.194195 1810 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election won for term 7
W20260812 19:08:06.194646 29457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 788) took 3947 ms (client timeout 1785 ms). Trace:
W20260812 19:08:06.194962 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 180883us
W20260812 19:08:06.196053 29457 rpcz_store.cc:269] 0812 19:08:02.246948 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.304376 (+ 57428us) service_pool.cc:227] Handling call
0812 19:08:06.194417 (+3890041us) spinlock_profiling.cc:243] Waited 3.86 s on lock 0x7b54002ff3d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:06.194548 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8879523584}
W20260812 19:08:06.196470 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 67374.2us
I20260812 19:08:06.197147 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.197394 29459 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:06.193401 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2267) took 4135 ms. Trace:
I20260812 19:08:06.197793 30685 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:06.198577 29575 rpcz_store.cc:276] 0812 19:08:02.057858 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.059223 (+ 1365us) service_pool.cc:227] Handling call
0812 19:08:02.059527 (+ 304us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.192610 (+4133083us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.193180 (+ 570us) spinlock_profiling.cc:243] Waited 4.13 s on lock 0x7b5400262854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:06.193343 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9503693056}
W20260812 19:08:06.181973 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 88755.3us
W20260812 19:08:06.187983 29567 rpcz_store.cc:269] 0812 19:08:04.724979 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.387530 (+662551us) service_pool.cc:227] Handling call
0812 19:08:06.182419 (+794889us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b5400135e54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:06.182533 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":463787904}
I20260812 19:08:06.191464 29450 rpcz_store.cc:276] 0812 19:08:04.675141 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:04.748780 (+ 73639us) service_pool.cc:227] Handling call
0812 19:08:04.749171 (+ 391us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.190315 (+1441144us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:06.190343 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:06.190383 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:06.190419 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.190899 (+ 480us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540038f254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:06.190942 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:06.191044 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3313920256}
I20260812 19:08:06.204461 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:06.038694 29491 debug-util.cc:398] Leaking SignalData structure 0x7b080025b160 after lost signal to thread 27445
W20260812 19:08:06.206372 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08002a98c0 after lost signal to thread 27450
I20260812 19:08:06.196836 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 804) took 1471 ms. Trace:
I20260812 19:08:06.206704 29342 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:08:06.205156 29452 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:05.904673 2239 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.207012 29576 rpcz_store.cc:276] 0812 19:08:04.725175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.129346 (+1404171us) service_pool.cc:227] Handling call
0812 19:08:06.196352 (+ 67006us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.196532 (+ 180us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:06.196558 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:06.196599 (+ 41us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:06.196637 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.196687 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:06.196787 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:05.689925 2275 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.206970 29179 catalog_manager.cc:5495] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260812 19:08:06.207787 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:06.208935 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2148) took 4131 ms. Trace:
I20260812 19:08:06.193315 2042 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.170379 30679 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:06.210781 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08002ff040 after lost signal to thread 29236
I20260812 19:08:06.210912 2132 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.165661 2071 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.223006 2071 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:06.204690 2212 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:06.224884 2212 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.225184 2001 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.131022 2123 raft_consensus.cc:729] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:06.228768 2195 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.231578 1478 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:06.218641 2262 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260812 19:08:05.624509 2252 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.163054 2215 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.162972 1722 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:06.214641 29459 rpcz_store.cc:276] 0812 19:08:02.077859 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:02.080761 (+ 2902us) service_pool.cc:227] Handling call
0812 19:08:02.081042 (+ 281us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:06.198318 (+4117276us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:06.208717 (+ 10399us) spinlock_profiling.cc:243] Waited 4.12 s on lock 0x7b54001ac554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:06.208880 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":9466406656}
I20260812 19:08:06.215459 2207 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:08:06.167467 1807 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.166683 2125 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:06.251466 2125 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.235009 2123 consensus_queue.cc:260] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.238528 2215 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.241991 2252 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.252390 2123 raft_consensus.cc:3037] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:06.254822 2215 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.256021 2199 raft_consensus.cc:3032] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:06.256390 2199 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:06.257077 2199 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.247180 1722 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.261864 1722 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.169006 2211 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.216280 29179 catalog_manager.cc:5495] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 1 to 2, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 2 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260812 19:08:06.212219 30674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 954) took 1785 ms (client timeout 1795 ms). Trace:
I20260812 19:08:06.178723 2145 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:06.220259 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08000fa720 after lost signal to thread 29379
I20260812 19:08:06.204244 2248 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.194651 1810 raft_consensus.cc:686] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:05.592550 2264 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:05.890497 2333 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.243430 30676 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:06.252391 2125 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.165764 2216 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.272186 2248 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.277729 2248 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.280251 2220 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:06.281104 2264 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.273741 2145 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:06.268469 30674 rpcz_store.cc:269] 0812 19:08:04.426767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:05.433403 (+1006636us) service_pool.cc:227] Handling call
0812 19:08:06.212151 (+778748us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:06.205680 2284 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:06.269505 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800331f60 after lost signal to thread 29381
W20260812 19:08:06.270609 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 125104us
I20260812 19:08:06.271060 29179 catalog_manager.cc:5495] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 4 to 6, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 6 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:08:06.286211 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800170160 after lost signal to thread 29496
W20260812 19:08:06.286569 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800170140 after lost signal to thread 29500
I20260812 19:08:06.280788 2220 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.286098 29179 catalog_manager.cc:5495] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 reported cstate change: term changed from 5 to 8, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 8 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: UNKNOWN } } }
W20260812 19:08:06.287698 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08003b8440 after lost signal to thread 29622
W20260812 19:08:06.287981 29491 debug-util.cc:398] Leaking SignalData structure 0x7b080031b200 after lost signal to thread 29623
I20260812 19:08:06.288491 2220 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:06.289134 29491 debug-util.cc:398] Leaking SignalData structure 0x7b0800370620 after lost signal to thread 30721
W20260812 19:08:06.289592 29491 debug-util.cc:398] Leaking SignalData structure 0x7b080032f8e0 after lost signal to thread 1820
I20260812 19:08:06.262920 2199 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.243054 2100 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.156243 2334 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.280794 1810 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.286314 29399 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.301085 29399 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:06.290522 29491 debug-util.cc:398] Leaking SignalData structure 0x7b08001b61c0 after lost signal to thread 2084
I20260812 19:08:06.298107 29338 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:06.308038 29399 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.307317 2333 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.309921 2195 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:06.310570 2100 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:06.311120 2100 raft_consensus.cc:686] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.311470 2334 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.311990 2100 consensus_queue.cc:237] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.314337 2073 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.310041 2284 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.319445 2073 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.325685 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.326308 29576 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:06.327435 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.328519 29567 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260812 19:08:06.328547 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.310252 2145 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.329236 29576 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:08:06.306746 2125 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:08:06.318239 2092 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.309900 2252 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:06.349092 30679 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:06.349720 1839 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.350210 1839 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:06.358693 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 169672us
I20260812 19:08:06.363495 29576 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:06.366375 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:06.367424 29576 raft_consensus.cc:2379] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:08:06.358197 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 230361us
I20260812 19:08:06.369164 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.373199 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.374032 29576 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:06.374615 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.375259 29567 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:06.338725 2284 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 19:08:06.376293 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 164293us
I20260812 19:08:06.378012 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.379096 29567 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:08:06.380044 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.381702 29567 raft_consensus.cc:2396] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260812 19:08:06.382478 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.382694 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.383363 29576 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 2.
I20260812 19:08:06.383782 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:06.386451 29399 connection.cc:232] RPC call timeout handler was delayed by -3.8057014239020646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260812 19:08:06.387385 29399 connection.cc:232] RPC call timeout handler was delayed by -2.6394884851761162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260812 19:08:06.387868 29399 connection.cc:232] RPC call timeout handler was delayed by -2.3343884851783514s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1925s for a response
W20260812 19:08:06.388211 29399 connection.cc:232] RPC call timeout handler was delayed by -2.1273884850088507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260812 19:08:06.388540 29399 connection.cc:232] RPC call timeout handler was delayed by -2.1102884851861745s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
W20260812 19:08:06.388854 29399 connection.cc:232] RPC call timeout handler was delayed by -2.0805884851142764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
W20260812 19:08:06.389163 29399 connection.cc:232] RPC call timeout handler was delayed by -2.0715884850360453s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260812 19:08:06.389485 29399 connection.cc:232] RPC call timeout handler was delayed by -2.0625884851906449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260812 19:08:06.382977 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.390818 29575 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:08:06.319864 2122 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.391014 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.393306 29338 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:06.384325 29574 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:06.370114 30629 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:06.391831 30623 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:06.391558 2352 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260812 19:08:06.378592 2216 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:06.402177 30623 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:06.402315 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 225540us
I20260812 19:08:06.399374 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:06.406507 30625 leader_election.cc:334] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.171s (SENT)
W20260812 19:08:06.407420 30625 leader_election.cc:334] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.196s (SENT)
I20260812 19:08:06.407858 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.408080 29282 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:06.390614 2125 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.411041 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.416831 29341 raft_consensus.cc:2379] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 9: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:06.420990 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.421489 29331 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260812 19:08:06.421983 2130 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:06.422351 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.422533 2207 consensus_queue.cc:1046] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:06.423091 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.423539 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.422371 2130 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.425688 2130 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:06.427034 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:06.428079 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.428692 29341 raft_consensus.cc:2354] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:08:06.429322 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:06.399299 1722 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:06.406790 2264 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:06.429803 29341 raft_consensus.cc:2362] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5. Re-sending same reply.
I20260812 19:08:06.467564 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.467841 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.469723 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.470468 2145 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:06.470985 2145 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.471398 2145 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:06.475369 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:06.403219 2212 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.430706 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.449738 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.430634 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.483091 29331 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:08:06.431000 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.432758 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:06.483258 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 75865.8us
W20260812 19:08:06.484141 30625 leader_election.cc:334] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.205s (SENT)
I20260812 19:08:06.422379 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.431406 29403 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:06.478852 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 280607us
I20260812 19:08:06.476802 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.485632 30625 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.482797 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.482805 29459 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:06.487823 29282 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:06.488083 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:06.488616 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55946.8us
W20260812 19:08:06.490653 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 59337.3us
I20260812 19:08:06.491372 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.491792 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.494423 29576 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:06.418502 1945 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:06.495075 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:06.495297 29403 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:06.407058 2118 consensus_queue.cc:1046] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260812 19:08:06.496096 29334 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:06.496371 29530 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:06.498442 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.498733 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.467090 2264 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.492012 29567 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:06.533942 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.540005 30675 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:06.536113 29281 connection.cc:232] RPC call timeout handler was delayed by -1.6343682419974357s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260812 19:08:06.540947 29281 connection.cc:232] RPC call timeout handler was delayed by -1.3076682419050485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19730000000000003s for a response
I20260812 19:08:06.540974 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.542198 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.540879 29334 raft_consensus.cc:595] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:06.543023 30671 raft_consensus.cc:2396] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260812 19:08:06.542806 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 46515.5us
I20260812 19:08:06.542378 30685 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:06.541792 30686 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:06.548647 29578 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:06.489434 1722 raft_consensus.cc:686] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:06.552091 29530 leader_election.cc:398] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:08:06.542845 29334 raft_consensus.cc:603] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:06.553192 29530 leader_election.cc:401] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:08:06.548086 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.553901 29576 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:06.554137 29530 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:06.494361 2248 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.490825 2349 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:06.542905 2092 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:06.565325 30684 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:06.566118 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.495707 2212 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:06.566190 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.418062 2125 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.568913 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.577706 30681 raft_consensus.cc:2396] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:06.564992 29530 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:06.502594 2219 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.499652 2262 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:08:06.581893 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.543725 2239 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:06.499708 1945 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.568615 2275 raft_consensus.cc:2781] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:06.550896 2264 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:06.477538 2215 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260812 19:08:06.550209 2352 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.575920 2212 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:06.568426 2042 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:08:06.504370 2004 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260812 19:08:06.579299 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 91295.1us
I20260812 19:08:06.508414 2100 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:06.572364 2092 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.543498 1807 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260812 19:08:06.586020 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.585299 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.586560 30675 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:06.539793 2344 consensus_queue.cc:1046] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260812 19:08:06.577354 2369 raft_consensus.cc:3032] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:06.597973 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:06.598987 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:06.599197 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.612993 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.609591 29578 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:06.599952 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:06.580852 2248 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260812 19:08:06.579166 2372 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.626986 2372 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.547652 2073 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260812 19:08:06.608850 2343 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.228s user 0.000s sys 0.001s
I20260812 19:08:06.583555 2122 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.586786 1945 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:06.608235 2092 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:06.654067 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.658738 29341 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
W20260812 19:08:06.659026 29527 leader_election.cc:334] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Timed out: RequestConsensusVote RPC to 127.26.205.68:41925 timed out after 1.927s (SENT)
I20260812 19:08:06.661187 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.610637 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.583762 2239 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.599545 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:06.614955 29577 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:06.672151 29580 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.672715 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.615387 30684 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:06.588174 2262 raft_consensus.cc:686] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.598825 2275 raft_consensus.cc:686] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:06.612885 2369 raft_consensus.cc:729] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:06.617488 2352 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.618570 2344 consensus_queue.cc:1225] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 19:08:06.584187 2219 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.619467 2248 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:06.619907 2100 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:06.620231 29399 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.615509 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.704859 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.625572 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.625764 1722 consensus_queue.cc:237] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.626469 30670 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:08:06.712432 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.594784 2333 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:06.597204 2004 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.597018 2212 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.632596 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.723343 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.723762 29578 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:06.601027 30685 raft_consensus.cc:1261] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:08:06.610181 29334 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "69bfa848552945a18495c13ca6707d65"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:06.639319 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.724498 29334 raft_consensus.cc:595] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:06.662339 29569 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:06.670863 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.677058 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.728024 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.728598 30681 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.729370 30671 raft_consensus.cc:2379] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:06.725687 29577 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:06.678010 2239 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:06.741179 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.745065 30681 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:06.746088 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.750420 30675 raft_consensus.cc:1261] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:06.623745 1839 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:06.704813 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 84670.9us
I20260812 19:08:06.753019 29399 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.705621 29332 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:06.722728 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:06.723337 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 181636us
I20260812 19:08:06.702899 29580 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:06.718966 2073 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:06.714449 2122 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.721197 2333 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:06.731431 2004 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:06.754966 30670 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:06.777273 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.756647 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.778708 29459 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:06.766498 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.780311 29457 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:08:06.711436 2248 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.769443 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:06.782135 29335 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:08:06.754945 30629 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:06.782320 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.755141 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:06.781734 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.783643 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.730684 2372 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.784636 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.779647 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.785583 29459 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.774005 29569 raft_consensus.cc:1261] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:06.787603 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.787833 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.788275 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.789067 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.783895 29453 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:06.790587 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.792613 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.793046 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.793220 29452 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:06.794003 29451 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:06.795189 29458 raft_consensus.cc:2379] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.749441 2252 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:06.798020 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.797394 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.799027 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.719854 2352 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.780140 2344 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election won for term 5
W20260812 19:08:06.815507 29527 leader_election.cc:334] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Timed out: RequestConsensusVote RPC to 127.26.205.68:41925 timed out after 2.012s (SENT)
I20260812 19:08:06.814909 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.815835 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.815528 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.817765 2262 consensus_queue.cc:237] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.818006 2248 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.780362 2333 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.782910 1810 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.775409 1807 consensus_queue.cc:1046] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260812 19:08:06.781594 2073 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.781741 2122 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.826372 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.874796 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.834721 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:06.835137 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 80063.3us
I20260812 19:08:06.877650 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:06.852815 29457 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.837975 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.848277 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.854388 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.849090 29527 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:06.881127 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:06.881557 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 66100.7us
I20260812 19:08:06.881754 29574 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:06.883155 29452 raft_consensus.cc:2379] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.883097 29527 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:06.849388 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.848140 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.834599 2219 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:06.846102 2275 consensus_queue.cc:237] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:06.846055 2284 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.842123 2368 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:06.838603 2130 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:06.858603 2364 log.cc:924] Time spent T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.246s user 0.001s sys 0.000s
W20260812 19:08:06.850485 2199 consensus_meta.cc:287] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.540s user 0.001s sys 0.005s
I20260812 19:08:06.838575 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:06.850078 30670 raft_consensus.cc:2396] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:06.854111 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.867108 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.868929 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.884439 30681 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
W20260812 19:08:06.825511 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 50141.4us
I20260812 19:08:06.904096 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.904598 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:06.826076 2125 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:06.898291 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.909575 30685 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:06.865744 2344 raft_consensus.cc:686] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:06.905126 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:06.905795 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:06.895882 1362 consensus_meta.cc:287] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.740s user 0.010s sys 0.003s
I20260812 19:08:06.906685 29334 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260812 19:08:06.906685 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:06.835608 2379 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:06.910614 29453 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.911077 29341 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:08:06.909829 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.866817 1945 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.942843 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.913623 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.943965 29452 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:06.909510 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.914505 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.945180 29458 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.930325 2125 consensus_queue.cc:1046] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260812 19:08:06.933456 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 68023.6us
I20260812 19:08:06.935411 2262 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:06.939708 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.951992 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.897738 2195 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:06.948559 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.894100 2334 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:06.866111 2264 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.825346 2145 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.956636 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.950956 29445 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:06.894662 2341 raft_consensus.cc:3032] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:06.953435 29342 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:06.942728 2379 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:06.947451 2123 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:06.908735 2071 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
W20260812 19:08:06.955904 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 51403.2us
I20260812 19:08:06.901535 2368 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:08:06.956321 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44878.9us
I20260812 19:08:06.955456 29453 raft_consensus.cc:2362] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7. Re-sending same reply.
I20260812 19:08:06.986256 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.990458 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.990674 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.793357 2369 consensus_queue.cc:260] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.007772 29399 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.010257 29452 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.010406 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.986282 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:07.011507 30685 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:06.947590 2042 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 4. Current term is 5.
I20260812 19:08:06.998836 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.012671 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:06.999594 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:06.943370 2400 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:07.030507 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.031466 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.032594 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:07.038679 29579 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:07.051263 2352 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:07.062606 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:07.016808 2392 log.cc:924] Time spent T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260812 19:08:06.955148 2262 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:06.955128 2195 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:07.004376 2379 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:06.991128 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.074370 29458 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
W20260812 19:08:06.992681 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 101632us
I20260812 19:08:06.871099 1810 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:06.909224 2284 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.013463 29445 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.013741 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.078444 29453 raft_consensus.cc:2362] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6. Re-sending same reply.
I20260812 19:08:07.013016 30681 raft_consensus.cc:2379] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:07.012130 2073 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.014592 2125 consensus_queue.cc:1046] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260812 19:08:07.015702 2130 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.015277 2344 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.016151 2215 consensus_queue.cc:1046] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:07.016556 1945 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.009145 2368 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.007256 2123 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.017485 2042 raft_consensus.cc:2741] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260812 19:08:07.017804 2122 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.015288 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.002012 2334 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:06.908305 2092 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.067436 2004 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:07.067708 2047 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:07.007362 2377 consensus_queue.cc:1046] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260812 19:08:07.067858 2327 consensus_queue.cc:1035] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260812 19:08:07.093125 2377 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:07.023273 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.023729 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.093448 29338 raft_consensus.cc:2354] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:07.068599 1986 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:07.003861 1807 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260812 19:08:07.063575 1362 consensus_queue.cc:1046] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260812 19:08:07.001888 2132 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:07.095348 2362 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:07.095755 2362 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:07.095552 2132 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.026407 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.096686 2362 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:07.099776 2398 log.cc:924] Time spent T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260812 19:08:07.077526 2195 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.073834 2262 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:07.102631 2405 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:07.103160 2373 raft_consensus.cc:3032] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:07.103595 2373 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:07.104365 2373 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:07.105499 2393 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260812 19:08:07.108173 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.108067 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:06.954599 29574 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260812 19:08:07.001473 2341 raft_consensus.cc:729] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:07.062523 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.063628 29457 raft_consensus.cc:2379] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.071360 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:07.007686 30686 raft_consensus.cc:2354] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8.
I20260812 19:08:07.073499 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:07.080348 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:07.081600 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.030293 2400 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:07.083343 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.062775 2219 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260812 19:08:07.085433 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 222785us
I20260812 19:08:07.085965 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:07.089686 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.158947 29580 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:07.026607 2071 consensus_queue.cc:1046] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260812 19:08:07.092219 2047 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:07.094297 1986 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:07.094925 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.026391 2411 consensus_queue.cc:1046] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260812 19:08:07.118028 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 102798us
W20260812 19:08:07.115790 29528 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.679s (SENT) [suppressed 42 similar messages]
I20260812 19:08:07.101758 2248 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:07.105791 2373 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:07.159437 29575 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:08:07.167150 30681 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:07.158915 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.167171 30623 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:07.178022 29453 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:07.167514 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:07.181936 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 298908us
I20260812 19:08:07.170217 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:07.158465 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 251840us
I20260812 19:08:07.171016 29578 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:07.110942 30685 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:07.173852 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.174242 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.174489 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:07.176146 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.214648 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.114450 2125 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:07.094389 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:07.172899 30679 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:07.169373 2284 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:07.180655 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 64894us
I20260812 19:08:07.117242 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:07.093467 2377 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:07.230171 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.230537 2377 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:07.231331 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.231667 30674 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:07.232448 2428 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:07.232867 2428 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:07.233233 2428 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.233755 2368 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.235940 2368 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:07.206687 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.237510 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.238126 29457 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 2.
I20260812 19:08:07.210848 30676 raft_consensus.cc:2396] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:07.211644 30681 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:07.216014 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.157768 2100 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:07.169337 2219 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:07.219856 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.213696 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.212634 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.214154 29445 raft_consensus.cc:2379] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.210843 29530 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.160444 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.173159 2047 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:08:07.209084 30625 leader_election.cc:334] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.766s (SENT)
I20260812 19:08:07.227451 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:07.271859 30685 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:07.272609 30625 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:07.165916 2400 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.232016 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.277504 29458 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.241705 30679 raft_consensus.cc:2379] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:07.241530 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.182842 2145 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.257974 30681 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 6. Ops: []
W20260812 19:08:07.259268 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 48548.7us
I20260812 19:08:07.228838 2125 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:07.258471 29452 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:07.262123 29576 raft_consensus.cc:1226] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:07.184978 2004 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.263303 29445 raft_consensus.cc:1226] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:07.205745 2412 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:07.271982 1810 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.231743 2132 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.165450 2239 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.177501 1986 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.298645 29445 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:07.103207 2405 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.275935 29459 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:07.238569 2092 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:07.281291 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 114202us
I20260812 19:08:07.277633 2284 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:08:07.288769 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:07.310422 29331 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:07.312176 29458 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:08:07.290616 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 78077.1us
I20260812 19:08:07.292225 2438 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:08:07.227897 2362 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:07.255790 2219 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.263019 2264 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.256480 2100 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:07.287130 1722 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:07.313508 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:07.296092 2412 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:07.315940 30623 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:07.387130 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 71264.4us
I20260812 19:08:07.388115 30623 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.319169 2239 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.322790 1986 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.325279 2283 raft_consensus.cc:3032] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
W20260812 19:08:07.273703 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 64667.6us
I20260812 19:08:07.297076 2125 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:07.365530 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 67114.1us
I20260812 19:08:07.304906 2284 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:07.398223 2284 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:07.336884 2438 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:07.287931 30676 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
W20260812 19:08:07.372957 2424 log.cc:924] Time spent T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.095s user 0.001s sys 0.000s
I20260812 19:08:07.379076 2379 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:07.385392 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 72083us
I20260812 19:08:07.388595 2417 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:07.391969 30623 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:07.393424 30625 leader_election.cc:334] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.815s (SENT)
I20260812 19:08:07.393991 2283 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:07.391607 2264 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:07.397673 1986 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.389322 2412 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.304204 1807 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260812 19:08:07.411046 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.443851 30670 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
W20260812 19:08:07.416550 29527 leader_election.cc:334] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925): Timed out: RequestConsensusVote RPC to 127.26.205.68:41925 timed out after 2.045s (SENT)
I20260812 19:08:07.420260 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:07.302925 2442 raft_consensus.cc:2741] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260812 19:08:07.426292 30625 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:07.395957 1722 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:07.415692 2438 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:07.444253 29527 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:07.444370 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:07.452737 2252 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:07.452361 2283 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.454246 2123 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.454818 2422 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:07.455061 2379 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.438958 2132 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:07.455852 2132 raft_consensus.cc:686] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:07.456442 2100 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:07.456748 2132 consensus_queue.cc:237] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:07.429430 2212 consensus_meta.cc:287] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.686s user 0.011s sys 0.001s
I20260812 19:08:07.415751 2405 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:07.464309 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 168877us
I20260812 19:08:07.461407 29567 raft_consensus.cc:2354] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:07.447184 2073 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:07.471349 29403 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:07.439298 2264 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:07.472837 29530 leader_election.cc:398] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:07.475777 29530 leader_election.cc:401] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:08:07.447448 1722 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:07.442934 1810 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260812 19:08:07.475353 2073 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:07.477491 2264 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:07.500762 2071 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:07.471410 30674 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:08:07.483376 1722 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.469859 2212 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.535876 2071 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:07.498801 2275 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.468994 2400 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:07.536700 2441 log.cc:924] Time spent T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.088s user 0.000s sys 0.001s
W20260812 19:08:07.716773 29453 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.536s user 0.006s sys 0.001s
W20260812 19:08:07.590720 2447 log.cc:924] Time spent T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.117s user 0.001s sys 0.000s
I20260812 19:08:07.504953 2073 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:07.741286 29450 raft_consensus.cc:1226] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:07.716877 1807 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:07.719538 2071 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:07.589368 2333 consensus_meta.cc:287] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.714s user 0.009s sys 0.000s
I20260812 19:08:07.589313 2195 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.731272 29576 raft_consensus.cc:1226] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: []
W20260812 19:08:07.785118 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 391704us
W20260812 19:08:07.785897 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 558513us
I20260812 19:08:07.788441 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:07.790872 29567 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:08:07.792554 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:07.793454 29331 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:07.796329 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:07.792578 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 376037us
I20260812 19:08:07.792775 29342 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:07.721791 29453 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:07.806011 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:07.805697 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:07.812611 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.814064 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.816304 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.816421 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.817323 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.818334 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:07.819008 29567 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.820591 29459 raft_consensus.cc:2379] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.820833 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.821521 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.820575 29342 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:08:07.826508 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.844429 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:07.845383 30670 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:07.846432 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.858651 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.859525 29452 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 9: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:07.804844 2100 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:07.863150 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:07.847113 29457 raft_consensus.cc:2362] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4. Re-sending same reply.
I20260812 19:08:07.753643 1807 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.870594 29567 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:07.864897 29572 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:07.767452 2333 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.879844 29459 raft_consensus.cc:2354] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:07.882428 29331 raft_consensus.cc:2454] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:07.882881 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:07.883635 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.887458 29458 raft_consensus.cc:3037] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:07.830132 2450 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:07.891434 30629 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:07.907608 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:07.908494 29572 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:07.909972 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:07.910740 29457 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:07.864305 2372 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:07.937469 29577 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
W20260812 19:08:07.896807 2219 consensus_meta.cc:287] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.526s user 0.007s sys 0.004s
I20260812 19:08:07.849068 2378 consensus_queue.cc:1046] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260812 19:08:07.952767 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:07.954138 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 149642us
I20260812 19:08:07.896041 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:07.956656 29528 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:07.958323 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 152694us
I20260812 19:08:07.953660 29458 raft_consensus.cc:2454] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
W20260812 19:08:07.959852 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 151428us
I20260812 19:08:07.963047 29282 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:07.963608 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 120067us
I20260812 19:08:07.964316 2122 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:07.964931 2122 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:07.964447 30623 leader_election.cc:398] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:07.969110 30623 leader_election.cc:401] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260812 19:08:07.959446 29528 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:07.971120 29527 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:07.972019 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:07.972285 2122 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:07.909387 2450 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.914924 2092 consensus_queue.cc:1046] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260812 19:08:07.979107 2450 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:07.977806 2252 consensus_meta.cc:287] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.525s user 0.008s sys 0.000s
I20260812 19:08:07.980595 2252 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:07.981937 2475 raft_consensus.cc:2781] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:08:07.953671 2262 consensus_meta.cc:287] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.784s user 0.011s sys 0.000s
I20260812 19:08:07.986802 2465 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:07.987546 2465 raft_consensus.cc:2726] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:07.988090 2465 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260812 19:08:08.026268 2248 consensus_meta.cc:287] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.841s user 0.007s sys 0.003s
W20260812 19:08:08.040218 2460 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.101s user 0.000s sys 0.000s
I20260812 19:08:08.021914 2262 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.043256 2239 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.043797 2239 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:07.983645 2212 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:07.954329 2219 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:08.029935 2457 log.cc:924] Time spent T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
I20260812 19:08:08.033555 2400 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.043702 2262 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:08.096107 2396 log.cc:924] Time spent T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
W20260812 19:08:08.095021 2373 consensus_meta.cc:287] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.888s user 0.010s sys 0.000s
I20260812 19:08:08.098630 2195 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:08.045985 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 150580us
I20260812 19:08:08.214409 29453 raft_consensus.cc:2396] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260812 19:08:08.227540 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 268074us
W20260812 19:08:08.227540 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 256349us
W20260812 19:08:08.192370 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 403831us
W20260812 19:08:08.120644 2220 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s user 0.016s sys 0.055s
W20260812 19:08:08.314810 2220 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s user 0.018s sys 0.055s
I20260812 19:08:08.315124 29527 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:08.315686 2362 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.955s user 0.004s sys 0.012s
I20260812 19:08:08.316952 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.316903 29339 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 10.
I20260812 19:08:08.296993 2219 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.137125 2333 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.261973 1807 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.319428 2219 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:08.204860 29446 raft_consensus.cc:2396] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260812 19:08:08.194751 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.134334 2475 raft_consensus.cc:686] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:08.272131 2047 consensus_meta.cc:287] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.985s user 0.010s sys 0.000s
I20260812 19:08:08.346303 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 813) took 1747 ms. Trace:
I20260812 19:08:08.297626 2252 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:08.346980 29579 rpcz_store.cc:276] 0812 19:08:06.598642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.598751 (+ 109us) service_pool.cc:227] Handling call
0812 19:08:08.345783 (+1747032us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b540009e054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.346029 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":930676480}
W20260812 19:08:08.348351 2412 consensus_meta.cc:287] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.900s user 0.007s sys 0.007s
I20260812 19:08:08.329869 2495 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.322662 2378 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:08.142627 2428 consensus_meta.cc:287] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.909s user 0.011s sys 0.000s
I20260812 19:08:08.351528 2378 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.351733 2428 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:08.157415 2438 consensus_meta.cc:287] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.690s user 0.007s sys 0.002s
I20260812 19:08:08.333916 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.354884 2438 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.356617 29458 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:08:08.346604 2047 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.360543 2047 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.360374 2475 consensus_queue.cc:237] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.363626 2239 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.364933 2239 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:08.367100 2378 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:08.367313 29446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41204 (request call id 817) took 1589 ms (client timeout 1775 ms). Trace:
W20260812 19:08:08.371073 29446 rpcz_store.cc:269] 0812 19:08:06.777789 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.816083 (+ 38294us) service_pool.cc:227] Handling call
0812 19:08:08.362353 (+1546270us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003bcc54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.367214 (+ 4861us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2990833920}
I20260812 19:08:08.369194 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:08.366765 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 172487us
I20260812 19:08:08.313226 2195 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.380208 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.381461 2239 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.318606 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.382375 2485 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:08.383832 2485 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260812 19:08:08.384040 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 65743.4us
W20260812 19:08:08.387168 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 70610.7us
I20260812 19:08:08.387287 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:08.382308 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 67450.9us
I20260812 19:08:08.387356 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.389196 29527 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.360707 2001 raft_consensus.cc:3032] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:08.389209 29332 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
W20260812 19:08:08.290218 2125 consensus_meta.cc:287] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.874s user 0.012s sys 0.001s
I20260812 19:08:08.392699 2125 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:08.387079 2264 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.670s user 0.011s sys 0.011s
I20260812 19:08:08.393910 2239 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:08.394662 2239 raft_consensus.cc:686] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:08.394256 2264 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.395172 2125 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.391947 2489 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.395831 2239 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.380716 2341 consensus_queue.cc:260] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.397403 2264 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.391427 2428 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.401294 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.257462 30674 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:08.398443 2341 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:08.403390 2377 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:08.348922 2412 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.215236 29456 raft_consensus.cc:1226] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:08.406103 29335 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:08.406121 29457 raft_consensus.cc:1226] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:08.407193 2412 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.350941 2495 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.407437 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:08.191126 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 221162us
I20260812 19:08:08.408172 2001 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:08.387686 2195 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.421767 2489 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:08.420020 2451 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573) with higher term. Message: Invalid argument: T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:08.434275 2122 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.434409 29335 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:08.429520 2352 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260812 19:08:08.435850 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:08.441497 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 55269.2us
W20260812 19:08:08.442255 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 55276.6us
W20260812 19:08:08.443079 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 54023.5us
W20260812 19:08:08.444866 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 43782.1us
W20260812 19:08:08.446820 30625 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.630s user 0.000s sys 0.020s
W20260812 19:08:08.446380 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 457991us
W20260812 19:08:08.450627 30625 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.664s user 0.000s sys 0.021s
I20260812 19:08:08.452314 2125 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.459178 2118 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:08.459744 2284 consensus_meta.cc:287] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.061s user 0.012s sys 0.005s
I20260812 19:08:08.456770 2001 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.451512 2438 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.447835 2475 consensus_queue.cc:1046] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260812 19:08:08.465265 2001 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.474440 2495 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:08.475550 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2202) took 1376 ms. Trace:
I20260812 19:08:08.475939 2195 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.477133 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.468891 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.469913 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.473253 2497 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:08.460007 2118 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.467396 2125 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.474057 2284 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.482188 29527 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.475950 29456 rpcz_store.cc:276] 0812 19:08:07.099460 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.296307 (+196847us) service_pool.cc:227] Handling call
0812 19:08:07.296536 (+ 229us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:08.406253 (+1109717us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.475235 (+ 68982us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b54002ff8d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:08.475451 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2108509440}
W20260812 19:08:08.478191 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 692204us
I20260812 19:08:08.490681 2489 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:08.491533 2118 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:08.482379 2497 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.494444 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.495654 2125 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:08.497522 2073 consensus_meta.cc:287] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.700s user 0.018s sys 0.000s
I20260812 19:08:08.497439 2497 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.498164 2073 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.498924 2497 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:08.495245 29337 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:08.499393 2465 raft_consensus.cc:3032] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:08.500635 2489 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:08.495991 2195 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260812 19:08:08.501209 1722 consensus_meta.cc:287] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.911s user 0.005s sys 0.015s
I20260812 19:08:08.501235 29452 raft_consensus.cc:2379] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:08.502395 2252 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.502054 1722 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.504144 2373 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260812 19:08:08.504745 1722 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.504834 1362 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.505728 1362 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.507699 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:08.509603 2132 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:08.510073 2132 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.510327 29403 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:08.511293 2125 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:08.502079 2489 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.511910 2125 raft_consensus.cc:686] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:08.512614 2369 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:08.501757 2465 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:08.514307 2125 consensus_queue.cc:237] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.515537 2073 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.517108 2465 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.518394 2465 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.520586 2132 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.522770 2132 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:08.523957 30629 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:08.528362 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2196) took 1498 ms. Trace:
I20260812 19:08:08.528791 29450 rpcz_store.cc:276] 0812 19:08:07.034411 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.287537 (+253126us) service_pool.cc:227] Handling call
0812 19:08:07.301737 (+ 14200us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:07.745740 (+444003us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.527911 (+782171us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400310454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:08.528181 (+ 270us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1010377472}
I20260812 19:08:08.530092 2372 consensus_queue.cc:1046] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:08.530414 2368 raft_consensus.cc:3032] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:08.531359 2368 raft_consensus.cc:729] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:08.531836 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 75590.9us
I20260812 19:08:08.532521 2368 consensus_queue.cc:260] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.533111 29281 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:08.533820 2368 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:08.534292 2379 consensus_queue.cc:1035] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:08:08.508054 1362 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.534642 2358 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:08.536769 2358 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:08.537204 2358 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:08:08.540185 2508 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260812 19:08:08.537285 2284 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:08.543277 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 935) took 1470 ms. Trace:
I20260812 19:08:08.543648 30676 rpcz_store.cc:276] 0812 19:08:07.072435 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.246772 (+174337us) service_pool.cc:227] Handling call
0812 19:08:07.287742 (+ 40970us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:07.415925 (+128183us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.542863 (+1126938us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b54006bd5d0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.543089 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":93523328}
W20260812 19:08:08.544952 2275 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.744s user 0.011s sys 0.007s
W20260812 19:08:08.549459 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 94020.5us
W20260812 19:08:08.547410 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 57528.2us
I20260812 19:08:08.560050 1810 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260812 19:08:08.571254 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 118380us
I20260812 19:08:08.572333 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.573292 2284 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.574664 29528 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:08.578025 2071 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.709s user 0.017s sys 0.007s
I20260812 19:08:08.578538 2071 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.579823 29334 raft_consensus.cc:2354] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:08:08.580799 2071 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.568764 2275 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
W20260812 19:08:08.586534 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 62782.7us
W20260812 19:08:08.592193 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 59233.8us
I20260812 19:08:08.566947 2195 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:08.599471 2453 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:08.600056 2453 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.600630 2453 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:08.600708 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:08.601102 29341 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:08.599607 2405 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:08.602206 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.600172 29451 raft_consensus.cc:2396] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260812 19:08:08.600021 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 139393us
I20260812 19:08:08.604971 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.606601 29445 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:08:08.619478 2412 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.620121 2412 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.624624 29326 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:08.627095 29574 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:08.620442 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 937) took 1537 ms. Trace:
I20260812 19:08:08.627792 30685 rpcz_store.cc:276] 0812 19:08:07.082670 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.293428 (+210758us) service_pool.cc:227] Handling call
0812 19:08:07.303692 (+ 10264us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:07.389718 (+ 86026us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:07.389752 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:07.389796 (+ 44us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:07.389830 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.619924 (+1230094us) spinlock_profiling.cc:243] Waited 85.8 ms on lock 0x7b540046df54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:08.620000 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:08.620241 (+ 241us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":197347200}
I20260812 19:08:08.630859 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 878) took 1070 ms. Trace:
I20260812 19:08:08.633260 2283 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.634195 2489 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.634820 2489 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.636427 2195 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.603982 2405 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.639667 29569 raft_consensus.cc:1226] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260812 19:08:08.615389 2284 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:08.640239 2284 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:08.640645 2284 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:08.638767 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2286) took 1899 ms. Trace:
I20260812 19:08:08.660387 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:08.657998 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.641880 29569 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 819) took 1790 ms. Trace:
I20260812 19:08:08.639808 2405 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.659000 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.655462 29341 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:08.664094 29451 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:08:08.658669 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.666436 29331 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
I20260812 19:08:08.664943 2405 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:08.664913 29342 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 3.
I20260812 19:08:08.658486 29339 raft_consensus.cc:1226] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:08.657176 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 821) took 1226 ms. Trace:
I20260812 19:08:08.656426 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.671373 29527 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.664204 29569 rpcz_store.cc:276] 0812 19:08:06.851824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.858664 (+ 6840us) service_pool.cc:227] Handling call
0812 19:08:08.639167 (+1780503us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:08.640113 (+ 946us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.641637 (+ 1524us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54000375d0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.641798 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4093821824}
I20260812 19:08:08.653501 30675 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:08.669526 29576 rpcz_store.cc:276] 0812 19:08:07.431037 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.431659 (+ 622us) service_pool.cc:227] Handling call
0812 19:08:07.432011 (+ 352us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:07.762141 (+330130us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.656670 (+894529us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400177d54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:08.656985 (+ 315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":687738240}
W20260812 19:08:08.678574 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 104010us
I20260812 19:08:08.668875 29577 rpcz_store.cc:276] 0812 19:08:06.738887 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.740944 (+ 2057us) service_pool.cc:227] Handling call
0812 19:08:08.638364 (+1897420us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000375d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.638589 (+ 225us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2733809408}
I20260812 19:08:08.682169 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:08.679623 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 92157.8us
I20260812 19:08:08.683437 30629 leader_election.cc:398] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:08:08.683728 30629 leader_election.cc:401] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:08:08.676064 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.598790 2125 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260812 19:08:08.686900 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.656404 2412 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.638789 2195 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.645488 2489 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.684479 2370 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:08.698911 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:08.643978 2453 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.698702 29337 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:08.706342 2211 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:08.707198 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.708381 29452 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260812 19:08:08.708871 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.710278 2486 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:08.710951 2369 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:08.712024 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.712831 2486 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:08.698750 2275 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:08.715374 2369 consensus_queue.cc:1046] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:08.718446 2500 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:08.721329 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:08.722044 30685 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
I20260812 19:08:08.723573 30676 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260812 19:08:08.724975 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:08.725730 30674 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:08.699896 2412 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:08.726500 2412 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:08.726895 2412 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:08.631279 29338 rpcz_store.cc:276] 0812 19:08:07.533038 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:07.533247 (+ 209us) service_pool.cc:227] Handling call
0812 19:08:08.517731 (+984484us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5800003398. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed642b0 00007f9c6ed4f7f9 00007f9c6ed4f2a2 00007f9c68441684
0812 19:08:08.518052 (+ 321us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:08.518368 (+ 316us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:08:08.553686 (+ 35318us) log.cc:842] Serialized 14 byte log entry
0812 19:08:08.594981 (+ 41295us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:08.595032 (+ 51us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:08.595083 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:08.595164 (+ 81us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:08.602878 (+ 7714us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":34591,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":6316,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":21931,"spinlock_wait_cycles":25555200,"thread_start_us":57345,"threads_started":2}
I20260812 19:08:08.729152 2073 consensus_queue.cc:1046] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 19:08:08.730414 30675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 958) took 1985 ms (client timeout 1699 ms). Trace:
I20260812 19:08:08.730703 2400 raft_consensus.cc:3032] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:08.731215 2400 raft_consensus.cc:729] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:08.732404 2400 consensus_queue.cc:260] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.733649 2400 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:08:08.730815 30675 rpcz_store.cc:269] 0812 19:08:06.744830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:06.781135 (+ 36305us) service_pool.cc:227] Handling call
0812 19:08:08.730180 (+1949045us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54006d9f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:08.730328 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4095322880}
I20260812 19:08:08.743067 29334 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:08:08.742393 29174 catalog_manager.cc:5495] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 4 to 6, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67). New cstate: current_term: 6 leader_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260812 19:08:08.744573 30625 leader_election.cc:334] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.638s (SENT)
I20260812 19:08:08.744987 29174 catalog_manager.cc:5495] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 6, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 6 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:08:08.747462 2486 raft_consensus.cc:3032] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:08.747905 2486 raft_consensus.cc:729] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.748767 2486 consensus_queue.cc:260] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:08.750358 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 59405.2us
W20260812 19:08:08.750815 30625 leader_election.cc:334] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.680s (SENT)
I20260812 19:08:08.751478 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.752008 2275 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:08.752885 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 154298us
W20260812 19:08:08.758282 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 46788.4us
I20260812 19:08:08.759115 2465 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:08:08.759464 29528 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.760046 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:08.760697 2485 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:08.761230 2485 raft_consensus.cc:686] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
W20260812 19:08:08.762017 2515 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260812 19:08:08.762398 2480 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:08.764171 2485 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.768723 29281 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:08.768913 2125 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:08.769656 2125 raft_consensus.cc:686] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:08.769899 29281 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.770517 1986 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:08.761262 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.770884 2125 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.771085 1986 raft_consensus.cc:686] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:08.772500 2465 raft_consensus.cc:2726] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:08.775137 2486 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:08.775403 1986 consensus_queue.cc:237] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:08.764099 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 92770.8us
I20260812 19:08:08.774549 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:08.775453 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 176987us
I20260812 19:08:08.795756 2195 consensus_queue.cc:1046] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:08:08.797429 2377 raft_consensus.cc:3032] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:08.797798 2377 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:08.798743 2377 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.801666 2497 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:08.804738 2377 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.806298 29334 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:08.807911 2369 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:08.809039 29528 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.812047 2465 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:08.812361 2400 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.812980 2400 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.815047 2400 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.815302 29527 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.802251 2497 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.819149 2252 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 19:08:08.819577 2252 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:08.819922 2252 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:08.820565 2497 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.821930 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.822358 2465 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:08.822976 2465 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.824864 2465 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.825551 2465 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.831466 2489 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:08.783715 2370 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6.
I20260812 19:08:08.833294 2333 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.841780 2333 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.845067 2333 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:08.841572 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 81687.5us
I20260812 19:08:08.839908 29174 catalog_manager.cc:5495] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 3, leader changed from 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 3 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
W20260812 19:08:08.850513 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47200.5us
I20260812 19:08:08.833667 2071 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:08.861778 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:08.864013 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:08.872951 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 103170us
I20260812 19:08:08.879570 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.880683 30623 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.882225 30623 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.883411 30623 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.884342 2500 raft_consensus.cc:2781] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:08.884918 2500 raft_consensus.cc:686] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.886312 2368 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:08.886057 2500 consensus_queue.cc:237] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.886724 2368 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.887115 2368 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:08.872344 2358 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.889003 2358 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.876106 2195 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260812 19:08:08.890545 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:08.869773 2400 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:08.856258 2465 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.873515 2071 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:08.875726 29174 catalog_manager.cc:5495] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 4, leader changed from a34c47dd29c14748a018d716ac6c39df (127.26.205.68) to 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65). New cstate: current_term: 4 leader_uuid: "6dd313c6592d4f33bab314a73f28b007" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:08.894604 30623 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.886137 2369 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:08.900236 29530 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:08.896653 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.901721 29335 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:08.869469 2377 consensus_queue.cc:1046] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260812 19:08:08.897819 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.904270 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:08.904256 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:08.894342 2071 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.897784 2369 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.905236 1986 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:08.919343 2400 consensus_queue.cc:1035] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260812 19:08:08.926437 2001 raft_consensus.cc:3032] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:08.926883 2001 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:08.924741 29403 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:08.928284 2073 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:08.927888 2001 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.912750 2071 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.928782 2073 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.939040 2073 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:08.928350 29403 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:08.929563 30625 leader_election.cc:334] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.780s (SENT)
I20260812 19:08:08.939182 29342 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:08.940920 2341 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:08.941371 2341 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:08.941752 2341 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:08.941592 2100 consensus_queue.cc:1046] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:08.942597 29338 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:08:08.914456 2333 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:08:08.943403 2333 raft_consensus.cc:686] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:08.915445 2453 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:08.942764 2100 raft_consensus.cc:3032] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Stepping down as leader of term 5
W20260812 19:08:08.942111 30625 leader_election.cc:334] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.798s (SENT)
I20260812 19:08:08.914922 2497 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:08.914834 2369 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:08.951562 30625 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.947858 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.954873 29527 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.955711 2412 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.956374 29527 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:08.961725 2465 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:08.963027 29335 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:08.964321 30629 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:08.965150 2453 raft_consensus.cc:2781] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:08.965683 2453 raft_consensus.cc:686] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:08.966010 1362 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:08.925021 2438 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260812 19:08:08.966548 1362 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:08.967252 2534 log.cc:924] Time spent T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260812 19:08:08.966590 2453 consensus_queue.cc:237] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.968886 1362 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.973227 2341 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:08.951293 1986 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:08.952476 2405 consensus_queue.cc:1046] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260812 19:08:08.951058 2497 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:09.005004 2497 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:08.975083 1807 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:09.006095 1807 raft_consensus.cc:686] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:09.007128 1807 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.008632 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:09.009383 29334 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:08.975791 2442 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260812 19:08:09.010079 1722 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:08.976548 2239 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:08.977499 2486 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:08.978390 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 57179.2us
I20260812 19:08:08.981124 2211 consensus_queue.cc:1046] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260812 19:08:08.948645 2100 raft_consensus.cc:729] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:09.001358 29335 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260812 19:08:08.999835 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.027129 29527 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:09.004331 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 74545.3us
I20260812 19:08:08.999641 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:08.957788 2412 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:08.979604 2275 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:09.050588 2211 raft_consensus.cc:3032] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:09.050287 2100 consensus_queue.cc:260] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.050978 2211 raft_consensus.cc:729] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.023824 2485 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:09.056772 2100 raft_consensus.cc:3037] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.057319 2465 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:09.056818 2211 consensus_queue.cc:260] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.061736 2211 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:09.062093 29399 leader_election.cc:398] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8.
I20260812 19:08:09.062412 29399 leader_election.cc:401] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260812 19:08:09.063519 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:09.064683 2412 consensus_queue.cc:1046] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260812 19:08:09.064965 2001 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.056818 29174 catalog_manager.cc:5495] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 4, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
W20260812 19:08:09.065514 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 123467us
W20260812 19:08:09.052228 29528 leader_election.cc:334] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.037s (SENT)
I20260812 19:08:09.068143 29528 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:08.998508 2252 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.090198 2125 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8.
I20260812 19:08:09.072446 2405 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:09.094159 2405 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:08.980861 2284 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.091750 2341 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.092417 2252 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:09.027770 2486 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:09.110832 30625 leader_election.cc:334] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.956s (SENT)
I20260812 19:08:09.109112 2195 consensus_queue.cc:1046] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260812 19:08:09.113063 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.065152 1986 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.114367 2284 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:09.116074 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 113587us
I20260812 19:08:09.096310 2485 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260812 19:08:09.117479 29527 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.119140 29527 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.119987 29577 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 2. Current term is 3. Ops: []
I20260812 19:08:09.120934 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:09.119833 2485 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:09.133096 29326 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 10.
I20260812 19:08:09.125988 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.128296 29174 catalog_manager.cc:5495] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 3, leader changed from 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 3 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:09.102761 2405 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:09.031658 2333 consensus_queue.cc:237] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.137238 29576 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:09.129554 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.121793 2239 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:09.157226 29574 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
W20260812 19:08:09.177368 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 131605us
I20260812 19:08:09.104511 30675 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89"
dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:35640
I20260812 19:08:09.183221 29174 catalog_manager.cc:5495] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:09.188057 30675 raft_consensus.cc:595] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.188530 30675 raft_consensus.cc:603] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:09.159982 2239 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:09.162297 2284 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:09.195802 29579 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.145862 2073 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.200733 1810 raft_consensus.cc:3032] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:09.194943 2239 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:09.190292 2195 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:09.213625 29528 leader_election.cc:334] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.106s (SENT)
I20260812 19:08:09.214113 29528 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:09.214224 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 142790us
I20260812 19:08:09.213042 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.223829 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:09.228561 29174 catalog_manager.cc:5495] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:09.237329 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.229830 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:09.228859 29580 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:09.239943 29174 catalog_manager.cc:5495] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 3 to 7, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66). New cstate: current_term: 7 leader_uuid: "182784ca1eb74135b8f16d8894fa4b01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } }
I20260812 19:08:09.207782 1810 raft_consensus.cc:729] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.242183 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:09.249375 29572 raft_consensus.cc:1261] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:09.203855 2368 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.254654 29174 catalog_manager.cc:5495] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c reported cstate change: term changed from 1 to 2, leader changed from dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67) to a34c47dd29c14748a018d716ac6c39df (127.26.205.68). New cstate: current_term: 2 leader_uuid: "a34c47dd29c14748a018d716ac6c39df" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:09.243441 1807 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 19:08:09.224306 2195 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:09.278223 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 70140.5us
I20260812 19:08:09.247052 2275 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:08:09.280923 27445 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260812 19:08:09.281723 2547 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.055s user 0.000s sys 0.001s
I20260812 19:08:09.282369 2368 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.283095 29179 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55498 (request call id 384) took 9220 ms. Trace:
I20260812 19:08:09.259951 2369 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.288606 2369 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.283871 27445 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260812 19:08:09.257632 2422 consensus_queue.cc:1046] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
I20260812 19:08:09.290251 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:08:09.290623 2422 raft_consensus.cc:3032] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Stepping down as leader of term 2
I20260812 19:08:09.290794 29575 raft_consensus.cc:1261] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260812 19:08:09.291067 2422 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.292026 2422 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.293360 2422 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 2 FOLLOWER]: Advancing to term 3
I20260812 19:08:09.297101 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.297726 29576 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:09.298694 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.300444 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:09.301291 2546 log.cc:924] Time spent T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.091s user 0.001s sys 0.000s
I20260812 19:08:09.303469 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:09.304095 30671 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:09.305001 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.306167 1810 consensus_queue.cc:260] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.308318 29580 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260812 19:08:09.306569 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.298348 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:09.315956 2497 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.316964 2475 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:09.317380 2475 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:09.317755 2475 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.276572 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:09.306627 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:09.278441 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 215011us
I20260812 19:08:09.329141 29574 raft_consensus.cc:2454] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:09.334437 29337 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:09.334252 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.318359 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.334659 30686 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:09.320379 29578 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:09.322088 29579 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:09.334604 29576 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:09.339609 30623 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:09.265372 2486 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:09.345631 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.347355 29326 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:09.345713 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:09.350376 29528 leader_election.cc:334] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.173s (SENT)
I20260812 19:08:09.347097 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:09.350864 29528 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.352413 30623 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:09.350311 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.354357 30686 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.356639 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.298735 2073 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:09.372661 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.334030 2497 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.359434 29339 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260812 19:08:09.283512 29179 rpcz_store.cc:276] 0812 19:08:00.017241 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:00.017521 (+ 280us) service_pool.cc:227] Handling call
0812 19:08:00.506930 (+489409us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b5400002dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93af0 00007f9c69e16f2a 00007f9c6fd68d6c 00007f9c6fe62c3e 00007f9c6e174fb4 00007f9c6e174f42 00007f9c6e174e71 00007f9c6e174dec 00007f9c6e1740a2 00007f9c6824494c 00007f9c68243d86
0812 19:08:09.225906 (+8718976us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:08:06.613157 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:08:06.651855 (+ 38698us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:08:06.651892 (+ 37us) write_op.cc:435] Acquired schema lock
0812 19:08:06.651906 (+ 14us) tablet.cc:660] Decoding operations
0812 19:08:06.746402 (+ 94496us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:08:06.746473 (+ 71us) write_op.cc:641] Partition lock acquired for write op
0812 19:08:06.746504 (+ 31us) tablet.cc:683] Acquiring locks for 9 operations
0812 19:08:06.747354 (+ 850us) tablet.cc:699] Row locks acquired
0812 19:08:06.747384 (+ 30us) write_op.cc:260] PREPARE: finished
0812 19:08:06.747543 (+ 159us) write_op.cc:270] Start()
0812 19:08:06.747668 (+ 125us) write_op.cc:276] Timestamp: P: 1786561686747490 usec, L: 0
0812 19:08:06.747692 (+ 24us) op_driver.cc:331] REPLICATION: starting
0812 19:08:06.748364 (+ 672us) log.cc:842] Serialized 3920 byte log entry
0812 19:08:07.281892 (+533528us) op_driver.cc:444] REPLICATION: finished
0812 19:08:07.448865 (+166973us) write_op.cc:301] APPLY: starting
0812 19:08:07.458556 (+ 9691us) tablet.cc:1365] starting BulkCheckPresence
0812 19:08:07.477038 (+ 18482us) tablet.cc:1368] finished BulkCheckPresence
0812 19:08:07.482665 (+ 5627us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:08:07.551599 (+ 68934us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:08:07.551839 (+ 240us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0812 19:08:07.551885 (+ 46us) write_op.cc:312] APPLY: finished
0812 19:08:07.552202 (+ 317us) log.cc:842] Serialized 71 byte log entry
0812 19:08:07.552375 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:07.552519 (+ 144us) write_op.cc:454] Released schema lock
0812 19:08:07.552565 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0812 19:08:07.552881 (+ 316us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:07.552913 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":57879,"reported_tablets":34,"spinlock_wait_cycles":176299776,"child_traces":[["op",{"apply.queue_time_us":157749,"apply.run_cpu_time_us":3901,"apply.run_wall_time_us":111049,"num_ops":9,"prepare.queue_time_us":243780,"prepare.run_cpu_time_us":25874,"prepare.run_wall_time_us":312671,"raft.queue_time_us":253098,"raft.run_cpu_time_us":31497,"raft.run_wall_time_us":108084,"replication_time_us":534083,"thread_start_us":568076,"threads_started":4,"wal-append.queue_time_us":277863,"wal-append.run_cpu_time_us":14787,"wal-append.run_wall_time_us":1741780}]]}
I20260812 19:08:09.374642 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.374745 30623 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:09.378919 2100 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 19:08:09.339330 1807 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:08:09.395756 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 45424.7us
W20260812 19:08:09.397825 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 48769.2us
I20260812 19:08:09.399214 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:09.399968 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.400486 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.406744 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.407469 29575 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.388196 2451 raft_consensus.cc:2781] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:09.345873 2368 consensus_queue.cc:1035] T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260812 19:08:09.411100 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.402375 29527 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.391407 2275 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:09.353609 2400 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260812 19:08:09.426157 2275 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:09.426928 29527 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.417549 29578 raft_consensus.cc:3037] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:09.414152 30686 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:09.396517 2486 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:09.421429 30676 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
W20260812 19:08:09.430956 29528 leader_election.cc:334] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 2.232s (SENT)
I20260812 19:08:09.434612 29528 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.420143 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.396951 2497 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:09.433091 30681 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:09.438097 29281 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.443043 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:09.443583 29579 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.443681 30686 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:09.428143 2275 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:09.435209 2369 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:09.453470 30674 raft_consensus.cc:2396] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260812 19:08:09.455302 1807 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.457304 2195 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:09.462694 2561 log.cc:924] Time spent T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260812 19:08:09.480587 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:09.493127 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 55123.7us
I20260812 19:08:09.493408 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.440251 2333 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:09.439304 2451 raft_consensus.cc:686] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.442703 2358 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:09.498407 29579 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:09.434131 2071 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.523574 29576 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 10: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:09.518451 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.524458 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.519975 29578 raft_consensus.cc:2454] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:09.533165 30686 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:09.536206 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.531764 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.522852 2486 raft_consensus.cc:686] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.481225 2489 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:09.547575 29574 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 10.
W20260812 19:08:09.547664 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 116735us
W20260812 19:08:09.547911 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 100915us
I20260812 19:08:09.555969 29334 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:09.556474 29569 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:09.527957 2358 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:09.488488 2369 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.560096 29337 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260812 19:08:09.548916 2379 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:09.555907 2465 raft_consensus.cc:2726] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:09.571573 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 59634us
I20260812 19:08:09.563063 2358 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:09.575266 29337 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:08:09.575986 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.578356 29576 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:09.579384 29574 raft_consensus.cc:1226] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260812 19:08:09.580871 29572 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:09.567312 2379 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:09.581645 29445 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:09.582949 29578 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:09.584167 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.556867 2489 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.589323 2369 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.591784 2239 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.594017 29281 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:09.596503 2001 consensus_meta.cc:287] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.531s user 0.010s sys 0.000s
I20260812 19:08:09.586750 2379 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:09.597919 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.598697 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.599402 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.599992 29445 raft_consensus.cc:1261] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:09.610049 29579 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:08:09.605739 30671 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
W20260812 19:08:09.583851 2572 log.cc:924] Time spent T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.102s user 0.000s sys 0.001s
I20260812 19:08:09.618474 29451 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:09.611202 29577 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:09.573776 2073 consensus_queue.cc:1046] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260812 19:08:09.676474 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.676752 29569 raft_consensus.cc:1261] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260812 19:08:09.677454 29452 raft_consensus.cc:2396] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260812 19:08:09.682370 30679 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:09.683914 29446 raft_consensus.cc:2379] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:09.615782 2195 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:09.613744 2497 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.695369 29281 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:09.616439 1722 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260812 19:08:09.701996 29281 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.579222 2451 consensus_queue.cc:237] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.711496 29281 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.712823 2372 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 19:08:09.713264 2372 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:09.713341 29453 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:09.713621 2372 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:09.714592 29452 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:09.715507 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.716313 29446 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:09.717088 2239 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:09.717980 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:09.718683 2475 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.710289 2594 consensus_queue.cc:1046] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260812 19:08:09.720834 2597 raft_consensus.cc:3032] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:09.721683 2597 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:09.723009 2597 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.724354 2597 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:09.618307 2486 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.726019 2489 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:09.712754 2283 raft_consensus.cc:2781] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:09.728044 2001 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:09.728081 29337 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:09.701387 2071 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:09.716214 2589 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260812 19:08:09.740342 29575 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:08:09.729363 2587 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.148s user 0.001s sys 0.000s
W20260812 19:08:09.749429 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 193161us
I20260812 19:08:09.748383 2073 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260812 19:08:09.752110 2601 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260812 19:08:09.758435 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.759933 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.761462 29580 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:09.763046 29453 raft_consensus.cc:2354] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4.
I20260812 19:08:09.765221 30623 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:09.764906 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.774720 29452 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:09.771951 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:09.781002 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.781780 29446 raft_consensus.cc:2379] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 3: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:09.767998 2475 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.712536 2497 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.777202 2001 raft_consensus.cc:686] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:09.798756 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:09.828562 29452 raft_consensus.cc:2379] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:09.777356 2283 raft_consensus.cc:686] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:09.830556 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 81608.2us
I20260812 19:08:09.837747 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:09.838127 2001 consensus_queue.cc:237] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.838786 29453 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
W20260812 19:08:09.843052 2509 log.cc:924] Time spent T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.658s user 0.005s sys 0.000s
I20260812 19:08:09.826071 1362 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260812 19:08:09.860450 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.864279 2195 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:09.866158 2372 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.867830 29579 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:09.867092 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.871663 29575 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.873068 30679 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:09.876592 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.877528 2377 raft_consensus.cc:3032] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:09.878294 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:09.878875 30676 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:09.879995 30676 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260812 19:08:09.877013 2372 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.882244 2377 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:09.882515 1362 consensus_queue.cc:1225] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer 182784ca1eb74135b8f16d8894fa4b01 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260812 19:08:09.883104 2611 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:09.883482 2611 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:09.883493 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.884409 29567 raft_consensus.cc:2396] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260812 19:08:09.884315 2611 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:09.836826 2422 consensus_meta.cc:287] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.543s user 0.008s sys 0.005s
I20260812 19:08:09.877523 30671 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:09.888042 2377 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.888382 2358 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.889179 2377 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:09.890463 2558 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.485s user 0.003s sys 0.000s
I20260812 19:08:09.891185 29577 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:08:09.826977 2475 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:09.892537 2475 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:09.892910 2475 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:09.894264 2125 raft_consensus.cc:3032] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:09.894655 2125 raft_consensus.cc:729] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:09.895462 2125 consensus_queue.cc:260] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:09.896656 2125 raft_consensus.cc:3037] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.897868 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:09.905894 29575 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:09.922369 29580 raft_consensus.cc:1226] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:09.923189 2379 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.899663 2358 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:09.935464 2604 log.cc:924] Time spent T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.101s user 0.000s sys 0.001s
W20260812 19:08:09.949266 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 184125us
I20260812 19:08:09.953608 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.954366 29575 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:09.961037 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.965516 29580 raft_consensus.cc:2379] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:09.978983 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:09.960008 2379 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:09.979395 29281 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:09.981727 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:09.988794 29575 raft_consensus.cc:2454] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:09.990510 30623 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:09.993634 29281 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:09.994872 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:09.995843 30671 raft_consensus.cc:2396] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260812 19:08:09.999552 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.000247 30671 raft_consensus.cc:2379] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 8: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:09.999375 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.006990 29580 raft_consensus.cc:2379] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 3: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:10.012645 30679 raft_consensus.cc:1226] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:09.998004 2613 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:09.950984 2283 consensus_queue.cc:237] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:09.998188 2071 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:10.014736 30686 raft_consensus.cc:1226] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:10.000952 2001 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:10.017118 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.018644 29281 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:10.017714 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.024469 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:10.019721 30679 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
W20260812 19:08:10.027998 2608 log.cc:924] Time spent T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260812 19:08:10.028340 2400 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:10.028961 2400 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.031224 2400 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.017354 2613 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:10.034657 2613 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.037546 2565 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.038941 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.039826 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.039963 2623 raft_consensus.cc:2781] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:10.037310 2400 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:10.042325 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.042691 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.043017 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.043617 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.080569 29575 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:10.080767 29574 raft_consensus.cc:2379] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:10.044386 29335 raft_consensus.cc:1261] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:08:10.044960 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.081663 29579 raft_consensus.cc:2379] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 10: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:10.045644 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.046708 29576 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260812 19:08:10.048053 29446 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:10.048650 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.083406 29580 raft_consensus.cc:1261] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:08:10.050653 2262 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:10.051600 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.054104 2475 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:10.055042 2575 log.cc:924] Time spent T c197fd0960e5419fafd6f7eec6f8959a P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.494s user 0.001s sys 0.000s
I20260812 19:08:10.035209 2617 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:10.062708 29527 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.015524 1807 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:10.072131 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 323450us
W20260812 19:08:10.072844 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 48413.7us
I20260812 19:08:10.079641 30679 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:10.049973 30629 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:10.086304 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.086992 30685 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:10.016564 2252 consensus_queue.cc:1035] T 5616609c9dad428f93e5627a7ca84f2a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260812 19:08:10.087258 2262 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:10.092234 2489 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 19:08:10.098400 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 46887.5us
I20260812 19:08:10.078766 2612 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:10.085089 2565 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:10.099150 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49308.2us
I20260812 19:08:10.101823 29281 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.103678 29527 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.097440 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.086864 2071 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.109540 29337 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:10.104100 2369 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:10.088238 2623 raft_consensus.cc:686] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:10.114481 29528 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.115665 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.095574 2377 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.127668 29281 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:10.131590 29530 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.135456 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.104321 2465 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:10.143174 30670 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:10.116772 2369 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.144790 2369 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.146130 2358 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:10.146680 2358 raft_consensus.cc:686] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.147615 2358 consensus_queue.cc:237] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.150955 29528 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:10.094215 2400 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.113477 2489 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:10.186951 30681 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:08:10.128911 2612 raft_consensus.cc:686] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.138034 2617 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:10.164497 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.192689 29452 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 11
I20260812 19:08:10.170972 2475 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.171351 2262 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.171867 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.174248 2341 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.221136 2341 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.223188 2341 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.174926 2565 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:10.175750 2626 log.cc:924] Time spent T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260812 19:08:10.176254 2252 raft_consensus.cc:2741] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260812 19:08:10.131291 2377 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.144757 2125 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.177564 29334 raft_consensus.cc:1226] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:10.178409 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.179183 2284 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:10.190675 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.193655 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.194552 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.195482 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.195698 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:10.197726 30670 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260812 19:08:10.154167 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:10.163754 30674 raft_consensus.cc:1226] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:10.218261 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.218483 2262 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:10.226293 29452 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 11.
I20260812 19:08:10.116528 1807 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.232561 30686 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:10.209848 2465 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:10.232200 30676 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Advancing to term 10
W20260812 19:08:10.237677 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 105861us
I20260812 19:08:10.212553 2617 raft_consensus.cc:686] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:10.231079 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 80208.6us
I20260812 19:08:10.240934 29528 leader_election.cc:398] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4.
I20260812 19:08:10.239559 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:10.242059 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 87984.2us
W20260812 19:08:10.242697 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 139081us
I20260812 19:08:10.231196 2613 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.246253 29326 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:10.246093 2284 consensus_queue.cc:1046] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:08:10.246989 2252 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:10.247380 2252 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:10.247421 29527 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.248278 2284 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:10.248775 2284 raft_consensus.cc:686] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:10.248890 2252 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.249657 2284 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.251245 2252 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.253985 2489 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.254551 2489 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.256458 2489 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.257339 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.259302 29341 raft_consensus.cc:1226] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:10.260838 2489 consensus_queue.cc:1046] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:08:10.261762 2252 raft_consensus.cc:3032] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:10.262241 2252 raft_consensus.cc:729] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:10.264051 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.263458 2252 consensus_queue.cc:260] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.265632 2252 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:10.264823 30671 raft_consensus.cc:2379] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:10.270859 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.271528 29576 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.275350 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.280059 29580 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:10.283157 29453 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260812 19:08:10.278309 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 149667us
I20260812 19:08:10.292204 29337 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:10.290385 29576 raft_consensus.cc:2454] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:10.271456 2594 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260812 19:08:10.297421 30623 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.298427 2489 consensus_queue.cc:1046] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:10.299379 2489 raft_consensus.cc:3032] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:10.299603 2489 raft_consensus.cc:729] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:10.300199 2489 consensus_queue.cc:260] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.301067 2489 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:08:10.304085 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 108473us
I20260812 19:08:10.304665 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.307487 29452 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:10.228305 2071 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.305363 29446 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:10.310273 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.310530 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.234580 2623 consensus_queue.cc:237] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.333853 29453 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:10.278669 1986 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:10.337179 29456 raft_consensus.cc:2396] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260812 19:08:10.314941 29457 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:10.291109 2372 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:10.343909 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.344787 29446 raft_consensus.cc:2354] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 9. Current term is 10.
I20260812 19:08:10.346910 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.347519 29446 raft_consensus.cc:2362] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8. Re-sending same reply.
I20260812 19:08:10.349298 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.350195 29450 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 10: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:10.352860 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.354734 1810 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:10.355726 2613 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.349601 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.298504 2637 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.277884 2125 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.352770 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.363435 29452 raft_consensus.cc:2354] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:10.358723 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.365993 29451 raft_consensus.cc:2362] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8. Re-sending same reply.
I20260812 19:08:10.304987 2405 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260812 19:08:10.353199 2400 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.350762 2372 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.311902 2001 consensus_queue.cc:1046] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260812 19:08:10.372958 29456 raft_consensus.cc:1261] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260812 19:08:10.374423 2253 raft_consensus.cc:2781] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:10.361267 1986 raft_consensus.cc:686] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.376680 1807 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.290159 2612 consensus_queue.cc:237] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.392918 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.378334 2073 consensus_queue.cc:1046] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:10.379348 2565 consensus_queue.cc:1046] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260812 19:08:10.381757 2640 log.cc:924] Time spent T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.087s user 0.000s sys 0.001s
I20260812 19:08:10.393069 2219 raft_consensus.cc:3032] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:10.402707 2219 raft_consensus.cc:729] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.372110 2617 consensus_queue.cc:237] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.403714 2219 consensus_queue.cc:260] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.382850 2637 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.397078 2377 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.405400 2611 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:10.406044 2637 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.392716 2600 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.406891 2600 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.408857 2600 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.409441 2475 consensus_queue.cc:1046] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:08:10.386468 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.410645 2475 raft_consensus.cc:3032] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:10.410987 2475 raft_consensus.cc:729] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:10.411288 29446 raft_consensus.cc:2396] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260812 19:08:10.397763 2565 raft_consensus.cc:3032] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:10.412273 2565 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:10.413076 2565 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.414290 2565 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.414570 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 989) took 1133 ms. Trace:
I20260812 19:08:10.414829 1986 consensus_queue.cc:237] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.414932 30675 rpcz_store.cc:276] 0812 19:08:09.281301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:09.286149 (+ 4848us) service_pool.cc:227] Handling call
0812 19:08:10.042186 (+756037us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:10.414363 (+372177us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540066f154. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:10.414513 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":778938496}
I20260812 19:08:10.382638 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.374791 2647 consensus_queue.cc:1046] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260812 19:08:10.362632 2649 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.364702 2601 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.393126 29453 raft_consensus.cc:1226] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:10.379878 2372 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.422003 2649 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.422101 2601 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.424273 2601 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.393805 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.424681 2649 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.425119 29452 raft_consensus.cc:2354] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:10.396543 29451 raft_consensus.cc:2379] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:10.397147 2073 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:10.425900 2073 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:10.426477 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.426030 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.414394 30684 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 3.
I20260812 19:08:10.426766 2073 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.427651 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 991) took 1145 ms. Trace:
I20260812 19:08:10.370339 2650 consensus_queue.cc:1035] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260812 19:08:10.395437 2368 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.428939 29282 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:10.429255 2368 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.430006 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.430641 29578 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:10.431303 2368 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.428041 2073 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.432560 30676 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 10.
I20260812 19:08:10.427975 30684 rpcz_store.cc:276] 0812 19:08:09.282053 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:09.283378 (+ 1325us) service_pool.cc:227] Handling call
0812 19:08:10.039585 (+756207us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b5800005498. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:10.427597 (+388012us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":53177088}
I20260812 19:08:10.422024 2647 raft_consensus.cc:3032] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:10.436241 2647 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:10.437022 2647 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.438053 2647 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:10.399044 2633 log.cc:924] Time spent T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.259s user 0.000s sys 0.001s
I20260812 19:08:10.401062 2645 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.411801 2475 consensus_queue.cc:260] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.404866 2125 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.419577 2253 raft_consensus.cc:686] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:10.419773 2369 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.443974 29459 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 3.
I20260812 19:08:10.442405 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:10.445391 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49649.3us
W20260812 19:08:10.387283 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 100179us
I20260812 19:08:10.457237 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.457705 29578 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:08:10.459159 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.404839 2219 raft_consensus.cc:3037] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:10.469004 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 50308.2us
I20260812 19:08:10.465862 29451 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:10.470345 29527 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.471875 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.472374 29527 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.474083 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.473743 29450 raft_consensus.cc:1261] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260812 19:08:10.464444 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.475718 29458 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:10.476562 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.472544 30676 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:10.477849 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.478340 29451 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.474768 29450 pending_rounds.cc:76] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 6
I20260812 19:08:10.454757 2613 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader election won for term 11
I20260812 19:08:10.482378 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.484452 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.484925 29456 raft_consensus.cc:2379] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:10.483556 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.458534 2650 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.486744 29282 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:10.487376 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.487646 2649 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:10.457697 2118 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260812 19:08:10.486136 1362 consensus_queue.cc:1046] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260812 19:08:10.488778 30685 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:10.485421 29578 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:10.490149 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.480538 29450 pending_rounds.cc:97] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260812 19:08:10.493515 29281 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:10.490694 2369 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:10.533986 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:10.500196 29458 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:10.535446 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.460717 2637 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.040s
I20260812 19:08:10.475136 2283 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:10.554540 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 64445.2us
I20260812 19:08:10.555927 29459 raft_consensus.cc:1261] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260812 19:08:10.555519 29450 raft_consensus.cc:2866] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:10.557603 29459 pending_rounds.cc:76] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 7
I20260812 19:08:10.557862 29459 pending_rounds.cc:97] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260812 19:08:10.558117 29459 raft_consensus.cc:2866] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:10.491113 2195 consensus_queue.cc:1046] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
W20260812 19:08:10.537616 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 50958.9us
W20260812 19:08:10.560554 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 67124.1us
I20260812 19:08:10.561722 2377 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:10.563023 2377 raft_consensus.cc:3032] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:10.563428 2377 raft_consensus.cc:729] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.496296 2118 consensus_queue.cc:1225] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer 182784ca1eb74135b8f16d8894fa4b01 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 19:08:10.572923 30623 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:10.596371 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 100965us
I20260812 19:08:10.481189 2252 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 19:08:10.597725 29453 raft_consensus.cc:1261] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260812 19:08:10.610016 30675 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:10.610844 29580 raft_consensus.cc:1226] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:10.573989 29451 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:10.598398 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.611539 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.564416 2377 consensus_queue.cc:260] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.613034 2377 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
W20260812 19:08:10.270447 2465 consensus_queue.cc:1159] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "182784ca1eb74135b8f16d8894fa4b01" responder_term: 7 status { last_received { term: 5 index: 7 } last_committed_idx: 7 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260812 19:08:10.464708 2645 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.614353 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.614632 29574 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:10.615319 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.615775 2645 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:10.615993 30674 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:10.610224 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:10.602916 30679 raft_consensus.cc:1261] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260812 19:08:10.553771 1362 consensus_queue.cc:1225] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer a34c47dd29c14748a018d716ac6c39df log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260812 19:08:10.607417 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.617502 30676 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.599581 2253 consensus_queue.cc:237] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.566160 2475 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.614952 29569 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:10.556702 2358 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:10.610194 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.498178 2613 raft_consensus.cc:686] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.555968 2649 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.628870 29579 raft_consensus.cc:1261] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:10.630978 29576 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.629293 29457 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:10.657187 2400 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.659507 29451 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260812 19:08:10.660068 2665 log.cc:924] Time spent T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.107s user 0.000s sys 0.000s
I20260812 19:08:10.659667 2400 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: NON_VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: true } }
I20260812 19:08:10.594537 2368 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
W20260812 19:08:10.632329 30625 leader_election.cc:334] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.565s (SENT)
I20260812 19:08:10.560313 2369 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260812 19:08:10.618534 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 45685us
I20260812 19:08:10.625643 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.652506 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:10.630697 2625 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.497404 2650 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.662103 29567 raft_consensus.cc:2379] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 4: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:10.665374 29527 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.655169 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:10.633400 2600 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:10.678128 29576 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:10.684913 30676 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:10.673147 29578 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
W20260812 19:08:10.674691 30625 leader_election.cc:334] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.775s (SENT)
I20260812 19:08:10.687285 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:10.687745 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.572731 2283 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.688614 29457 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:10.656366 2358 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.692631 29579 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260812 19:08:10.633344 2252 raft_consensus.cc:686] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:10.613125 2610 consensus_queue.cc:1035] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260812 19:08:10.655674 2648 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.663733 2369 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.630832 2195 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.061s
I20260812 19:08:10.633983 2071 consensus_queue.cc:1046] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260812 19:08:10.690073 2600 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.718930 2600 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.719820 2601 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260812 19:08:10.714669 30625 leader_election.cc:334] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.835s (SENT)
I20260812 19:08:10.689939 2625 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:10.723711 30625 leader_election.cc:334] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659): Timed out: RequestConsensusVote RPC to 127.26.205.66:43659 timed out after 1.914s (SENT)
I20260812 19:08:10.724151 2625 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.677886 2612 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:10.689035 2465 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:10.710392 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.714607 2649 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.715651 2451 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.699173 2358 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.633814 2660 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260812 19:08:10.716536 2594 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:10.716357 2252 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.708249 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.731431 29281 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.734632 2400 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.735141 2613 consensus_queue.cc:237] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.739216 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.731289 2657 consensus_queue.cc:1035] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260812 19:08:10.742512 29459 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260812 19:08:10.743036 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.749495 2372 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.743410 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.744118 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.744447 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.745170 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.802852 29446 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 10: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:10.725036 2648 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.746595 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.747454 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.744025 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.804019 29576 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:10.802204 29453 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:10.804193 29458 raft_consensus.cc:2454] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:10.747205 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.726389 2369 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:10.757604 29578 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:10.806102 30623 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.772485 29569 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 11.
I20260812 19:08:10.806761 30625 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:10.796314 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:10.797865 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.761997 2650 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.730581 2612 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.732370 2465 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.801019 29580 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.745486 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.857138 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.802816 29456 raft_consensus.cc:2362] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6. Re-sending same reply.
I20260812 19:08:10.856570 29574 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260812 19:08:10.617511 27445 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260812 19:08:10.824438 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:10.858004 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 126674us
I20260812 19:08:10.861155 30676 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:10.809801 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.791832 29457 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:10.759519 2219 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:10.817704 29326 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:10.813565 29445 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:10.809428 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:10.856829 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.862262 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.865077 29453 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:10.862589 29332 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:10.862917 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.860119 29452 raft_consensus.cc:2362] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7. Re-sending same reply.
I20260812 19:08:10.865329 29575 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:10.866640 29578 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:08:10.865211 29576 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:10.869406 29281 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:10.876327 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.878873 2489 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.879426 2489 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.853605 2596 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:10.854148 2625 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:10.881464 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 74779.4us
I20260812 19:08:10.811259 2648 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.890516 30625 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:10.867137 2660 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:10.818496 2283 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.818635 2372 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.855234 2645 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:10.879495 2649 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.863538 2422 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:10.859091 2612 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:10.864010 2465 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:10.890105 29282 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:10.895398 29281 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:10.799075 1362 consensus_queue.cc:1035] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260812 19:08:10.880445 2219 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.798177 2451 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.885406 2596 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.905143 2422 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:10.908476 2422 consensus_queue.cc:1035] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:10.909183 2596 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.910077 2687 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:10.910616 2687 raft_consensus.cc:686] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.911178 2219 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.911291 2451 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.911464 2687 consensus_queue.cc:237] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.906100 2611 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:10.912983 2611 raft_consensus.cc:686] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.913622 29342 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260812 19:08:10.903035 2660 raft_consensus.cc:686] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.914855 2660 consensus_queue.cc:237] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.913555 29341 raft_consensus.cc:3032] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
W20260812 19:08:10.901715 2690 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.093s user 0.001s sys 0.000s
I20260812 19:08:10.925570 29341 raft_consensus.cc:729] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:10.929348 29567 raft_consensus.cc:1261] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260812 19:08:10.904624 2283 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:10.930655 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.931247 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.913808 2611 consensus_queue.cc:237] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:10.933001 2676 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.218s user 0.000s sys 0.005s
I20260812 19:08:10.896306 2688 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260812 19:08:10.893888 2625 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.946108 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:10.903488 2645 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.951566 2625 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.952194 2645 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:10.952482 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.953125 2649 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:10.945205 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:10.953260 2369 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.946583 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:10.954957 29342 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 10.
I20260812 19:08:10.955552 2369 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.954952 30679 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:10.963265 2372 consensus_queue.cc:1035] T 0e4ba354bddd43ec97a890c3bb577330 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:10.926355 2489 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:10.941730 29575 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:10.965093 2649 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:10.937573 2358 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:10.972405 2358 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.973227 29334 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:10.971728 2600 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.978264 2600 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:10.984826 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:10.985534 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:10.986182 29451 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:08:10.997232 29580 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:10.946225 29341 consensus_queue.cc:260] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:10.993238 29451 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:11.000653 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:11.000852 2649 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 19:08:11.001307 2649 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:10.940097 2283 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:11.001341 29575 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:11.013135 29282 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:11.001845 2649 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:11.026721 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:11.027387 29332 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
W20260812 19:08:11.025986 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 133518us
I20260812 19:08:11.028501 30629 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:11.034605 30625 leader_election.cc:398] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7.
I20260812 19:08:10.943600 2623 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260812 19:08:11.025796 2283 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:11.029337 1986 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
W20260812 19:08:11.062224 2690 log.cc:924] Time spent T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260812 19:08:11.074118 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:11.074978 30676 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:11.076721 30676 raft_consensus.cc:1261] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:11.078562 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:11.060778 2703 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:11.070744 2489 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:11.069722 2625 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:11.074388 1986 raft_consensus.cc:686] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:11.092940 29527 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:11.096078 29575 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:11.083575 2703 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:11.106938 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:11.108855 30676 raft_consensus.cc:2379] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:11.108882 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:11.110069 29567 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:11.115087 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:11.115790 29578 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:11.118474 29527 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:11.101404 2284 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:11.124321 30685 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:11.124637 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:11.127583 2284 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:11.128062 2284 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:11.131248 1986 consensus_queue.cc:237] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:11.142705 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:11.152192 2675 log.cc:924] Time spent T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.495s user 0.001s sys 0.002s
I20260812 19:08:11.177390 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:11.178311 29580 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
W20260812 19:08:11.181867 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 152802us
I20260812 19:08:11.108633 2703 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:11.191546 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:11.192101 30625 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:11.194147 2611 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:11.195534 2657 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:08:11.198254 2649 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:11.191098 2613 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:11.198891 2369 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 19:08:11.199685 2715 log.cc:924] Time spent T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.088s user 0.001s sys 0.000s
I20260812 19:08:11.203577 2649 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 11 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:11.204193 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:11.204805 29332 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:11.215637 29332 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 11.
I20260812 19:08:11.124796 2645 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:11.217614 30685 raft_consensus.cc:1261] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:11.192056 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:11.100524 2489 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:11.223804 29530 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:11.217782 2718 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:11.221262 29569 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
W20260812 19:08:11.226378 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 422457us
I20260812 19:08:11.105865 2625 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:11.244406 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:11.252429 2400 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:11.252702 2625 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:11.253054 2400 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:11.255324 2400 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:11.190866 2372 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260812 19:08:11.257473 2489 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:11.258692 30625 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:11.258966 2489 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Leader election won for term 11
I20260812 19:08:11.259420 2489 raft_consensus.cc:686] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:11.261163 2706 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:11.224110 2252 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260812 19:08:11.262477 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:11.286691 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:11.295774 29342 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 10. Current term is 11. Ops: []
I20260812 19:08:11.297027 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:11.301903 2697 log.cc:924] Time spent T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
W20260812 19:08:11.302412 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 43801.7us
I20260812 19:08:11.307773 2650 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260812 19:08:11.310281 2707 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:11.242085 2718 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:11.192591 2610 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260812 19:08:11.316002 2718 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:11.316663 2687 consensus_queue.cc:1035] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:08:11.283169 2358 consensus_queue.cc:1035] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260812 19:08:11.326012 2283 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:11.331403 2647 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:11.331928 2647 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:11.332319 2647 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:11.301818 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:11.530645 2734 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:11.531144 2734 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:11.532768 2734 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:11.563920 29453 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:11.565404 2283 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:11.789532 2734 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:11.790302 2734 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:11.792994 2734 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:12.117105 2734 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:12.117700 2734 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:12.119786 2734 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:12.403039 2734 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:12.473057 2612 consensus_meta.cc:287] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 1.567s user 0.008s sys 0.012s
I20260812 19:08:12.475854 2734 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:12.779208 2740 consensus_queue.cc:1046] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:12.799510 2734 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:12.800218 2734 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:12.800691 2734 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:12.822264 2734 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:12.822856 2734 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:12.823438 2734 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:12.825549 2734 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:12.671043 2612 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:12.828488 2612 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:12.853482 29453 consensus_meta.cc:287] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.287s user 0.008s sys 0.000s
W20260812 19:08:12.984851 29580 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.767s user 0.016s sys 0.020s
W20260812 19:08:12.985605 29580 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.768s user 0.017s sys 0.020s
W20260812 19:08:13.076498 2660 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.857s user 0.010s sys 0.023s
W20260812 19:08:13.077085 2660 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.857s user 0.010s sys 0.024s
I20260812 19:08:13.078557 2740 raft_consensus.cc:3032] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Stepping down as leader of term 10
I20260812 19:08:13.079036 2740 raft_consensus.cc:729] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:13.079840 2740 consensus_queue.cc:260] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.081072 2740 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:13.082818 2741 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.083395 2741 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.088466 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2342) took 1890 ms. Trace:
W20260812 19:08:13.089453 29400 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.000s sys 0.004s
W20260812 19:08:13.089908 29400 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.605s user 0.000s sys 0.005s
I20260812 19:08:13.090190 2747 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.090833 2747 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.088923 29580 rpcz_store.cc:276] 0812 19:08:11.197649 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.197809 (+ 160us) service_pool.cc:227] Handling call
0812 19:08:11.198175 (+ 366us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:12.986093 (+1787918us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:12.986129 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:12.986721 (+ 592us) log.cc:842] Serialized 26 byte log entry
0812 19:08:13.088101 (+101380us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:13.088140 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.088212 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.088261 (+ 49us) raft_consensus.cc:1671] finished
0812 19:08:13.088271 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.088403 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18734,"raft.queue_time_us":59573,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":1869636,"threads_started":2,"wal-append.queue_time_us":99234}
W20260812 19:08:13.093753 29530 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.869s user 0.008s sys 0.019s
W20260812 19:08:13.094118 29530 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.870s user 0.008s sys 0.019s
W20260812 19:08:13.094396 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 1.87055e+06us
I20260812 19:08:13.095335 2747 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.101558 2734 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:11.298377 2489 consensus_queue.cc:237] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.102208 2734 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.105240 2747 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.106770 2741 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.124425 2747 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.134392 2741 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.135573 2741 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.126667 2756 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.076519 2742 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.137985 2756 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.138513 2742 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.140604 2742 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.141242 2742 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.141361 2756 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.107582 2744 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.128016 2489 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.142360 2489 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.142256 2744 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.143882 2741 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.144521 2489 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.122495 2645 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:13.131072 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 1.82941e+06us
I20260812 19:08:13.145696 2645 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:13.146562 29445 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39326 (request call id 2248) took 2723 ms (client timeout 1595 ms). Trace:
W20260812 19:08:13.132143 29567 consensus_meta.cc:287] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.022s user 0.006s sys 0.018s
W20260812 19:08:13.146942 29445 rpcz_store.cc:269] 0812 19:08:10.422674 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:10.425760 (+ 3086us) service_pool.cc:227] Handling call
0812 19:08:13.146351 (+2720591us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54000c67d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:13.146469 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":745352576}
I20260812 19:08:13.147346 29567 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
W20260812 19:08:13.147825 29567 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 886) took 2039 ms (client timeout 1644 ms). Trace:
W20260812 19:08:13.148164 29567 rpcz_store.cc:269] 0812 19:08:11.108448 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.108646 (+ 198us) service_pool.cc:227] Handling call
0812 19:08:13.147761 (+2039115us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:13.116180 2740 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.132606 2648 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.104321 2734 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.136075 2747 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.157707 2645 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.123934 2750 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.127319 2752 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.159143 2752 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.150873 2740 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.161515 2752 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.144878 2744 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.141734 2742 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.151149 2648 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:13.156729 2465 consensus_meta.cc:287] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.250s user 0.008s sys 0.015s
I20260812 19:08:13.164095 2742 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.164742 2465 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.166162 2750 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.167222 2465 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.168107 2750 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.164057 2648 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.162222 2740 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.179838 2648 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.182755 2648 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.185177 2648 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.181385 2465 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.180873 2734 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:13.188233 2617 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.973s user 0.015s sys 0.030s
W20260812 19:08:13.188694 2617 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.974s user 0.015s sys 0.030s
I20260812 19:08:13.186196 2465 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:13.190558 29572 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.007s user 0.010s sys 0.017s
W20260812 19:08:13.191080 29572 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.011s user 0.012s sys 0.017s
I20260812 19:08:13.191882 2465 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:13.195071 2284 consensus_meta.cc:287] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 2.067s user 0.019s sys 0.010s
I20260812 19:08:13.199007 29453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2265) took 1974 ms. Trace:
I20260812 19:08:13.195210 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2341) took 2017 ms. Trace:
I20260812 19:08:13.199144 2284 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.199393 29453 rpcz_store.cc:276] 0812 19:08:11.224799 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.224956 (+ 157us) service_pool.cc:227] Handling call
0812 19:08:11.248717 (+ 23761us) raft_consensus.cc:1457] Updating replica for 3 ops
0812 19:08:11.284130 (+ 35413us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.284166 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0812 19:08:11.562699 (+278533us) log.cc:842] Serialized 616 byte log entry
0812 19:08:11.563792 (+ 1093us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:12.854197 (+1290405us) log.cc:842] Serialized 14 byte log entry
0812 19:08:13.133136 (+278939us) log.cc:842] Serialized 14 byte log entry
0812 19:08:13.198692 (+ 65556us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.198769 (+ 77us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.198816 (+ 47us) raft_consensus.cc:1671] finished
0812 19:08:13.198825 (+ 9us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.198943 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":16027,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":38149,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":18578,"thread_start_us":344102,"threads_started":4,"wal-append.queue_time_us":271964,"wal-append.run_cpu_time_us":2508,"wal-append.run_wall_time_us":1008392}
I20260812 19:08:13.201797 2284 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.187235 2734 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.199548 29572 rpcz_store.cc:276] 0812 19:08:11.177479 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.179611 (+ 2132us) service_pool.cc:227] Handling call
0812 19:08:11.179935 (+ 324us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:13.191408 (+2011473us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:13.191444 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:13.191896 (+ 452us) log.cc:842] Serialized 26 byte log entry
0812 19:08:13.193454 (+ 1558us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:13.193487 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.193557 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.195013 (+ 1456us) raft_consensus.cc:1671] finished
0812 19:08:13.195032 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.195156 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":45852,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":7214,"thread_start_us":2012249,"threads_started":2,"wal-append.queue_time_us":1586}
W20260812 19:08:13.204128 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08002cad40 after lost signal to thread 27450
I20260812 19:08:13.205562 2734 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:13.218350 29326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 921) took 2724 ms (client timeout 3010 ms). Trace:
W20260812 19:08:13.218827 29326 rpcz_store.cc:269] 0812 19:08:10.494444 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:10.494582 (+ 138us) service_pool.cc:227] Handling call
0812 19:08:13.218102 (+2723520us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54002323d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:13.218259 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":649375488}
W20260812 19:08:13.221284 2601 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.007s user 0.009s sys 0.016s
W20260812 19:08:13.221781 2601 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.008s user 0.009s sys 0.016s
W20260812 19:08:13.227752 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.928s user 0.012s sys 0.026s
W20260812 19:08:13.228307 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.929s user 0.013s sys 0.026s
W20260812 19:08:13.228850 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 1.93178e+06us
I20260812 19:08:13.230168 29399 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:13.231719 2760 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 19:08:13.232338 2760 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:13.233006 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.233218 2760 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.234338 2750 consensus_queue.cc:1035] T 75e1d0973f2d43b6affb3b455c7789a1 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:13.235291 2740 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:13.236253 29399 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.237486 2742 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:13.238422 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.239296 2742 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:13.240489 29399 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:13.241287 2742 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:13.241765 2742 raft_consensus.cc:686] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:13.242733 2742 consensus_queue.cc:237] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:13.246685 2647 consensus_meta.cc:287] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 1.914s user 0.021s sys 0.012s
I20260812 19:08:13.247251 2647 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.248142 29399 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.249475 2647 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.250607 2761 raft_consensus.cc:2726] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:13.252063 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.252643 2761 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.255033 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.257542 29399 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.258724 2740 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 19:08:13.260349 2740 raft_consensus.cc:2726] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:13.253805 2761 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.264145 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:13.264823 29569 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 5
I20260812 19:08:13.264376 2761 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:13.267627 30675 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.952s user 0.024s sys 0.010s
W20260812 19:08:13.268260 30675 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.952s user 0.025s sys 0.011s
I20260812 19:08:13.269004 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 973) took 2046 ms. Trace:
I20260812 19:08:13.258296 2647 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:08:13.269338 30675 rpcz_store.cc:276] 0812 19:08:11.222919 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.223118 (+ 199us) service_pool.cc:227] Handling call
0812 19:08:11.242500 (+ 19382us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:11.242864 (+ 364us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.242891 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:11.243316 (+ 425us) log.cc:842] Serialized 26 byte log entry
0812 19:08:13.268613 (+2025297us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:13.268654 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.268727 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.268772 (+ 45us) raft_consensus.cc:1671] finished
0812 19:08:13.268783 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.268923 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":4283,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":47133,"thread_start_us":1952498,"threads_started":1,"wal-append.queue_time_us":143912,"wal-append.run_cpu_time_us":1781,"wal-append.run_wall_time_us":1003994}
I20260812 19:08:13.271956 29399 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:13.272972 29528 connection.cc:232] RPC call timeout handler was delayed by -1.6103938561864197s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.192s for a response
W20260812 19:08:13.273468 29528 connection.cc:232] RPC call timeout handler was delayed by -1.0938148400746286s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260812 19:08:13.274044 29399 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.275735 2761 raft_consensus.cc:2726] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:13.276316 2761 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260812 19:08:13.276844 2761 consensus_queue.cc:1035] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 19:08:13.277135 2734 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.278019 2734 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.278431 29399 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.279263 2742 raft_consensus.cc:2726] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:13.280573 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.281251 2742 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:13.271665 29569 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:13.281693 2742 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.282114 2742 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:13.276746 2465 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.288019 2465 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.290588 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.281015 2284 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.294670 2284 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.295115 2742 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.298007 2647 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:13.298695 2647 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.299310 2647 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:13.302589 29572 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260812 19:08:13.304226 2704 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.305050 2623 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.305583 2625 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.306217 2757 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.306691 2625 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.307677 2707 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.308828 1722 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.306233 2368 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.309182 2707 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.310825 2625 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.310555 2368 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.311928 2625 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.312634 2625 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.307118 2405 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.315253 1722 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.316134 2405 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.316577 2707 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.321228 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.307056 2757 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.305075 2704 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.303398 2734 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.323925 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.325350 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.326133 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 923) took 2554 ms. Trace:
I20260812 19:08:13.328286 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.330258 29457 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.330179 29338 rpcz_store.cc:276] 0812 19:08:10.763171 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:10.763321 (+ 150us) service_pool.cc:227] Handling call
0812 19:08:10.800083 (+ 36762us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:10.911731 (+111648us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:10.911766 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:10.911806 (+ 40us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:10.911835 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.317235 (+2405400us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400231ed4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:13.317289 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.317427 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":256331648}
I20260812 19:08:13.331382 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.334100 29457 raft_consensus.cc:2379] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 4: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:13.334614 2772 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.335018 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.335122 2772 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.335646 29457 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.336668 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.337906 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.338392 2772 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.339126 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.340250 29457 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:13.341209 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.341647 29459 raft_consensus.cc:1261] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:13.342041 29446 raft_consensus.cc:2396] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260812 19:08:13.343196 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.345041 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.339232 2772 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.344209 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.345696 29456 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.325886 2760 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:13.347816 2747 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.348796 2744 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.357764 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.358604 29459 raft_consensus.cc:2396] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260812 19:08:13.349908 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.350551 30623 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.351197 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.343312 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.359148 29446 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:13.360821 29450 raft_consensus.cc:2396] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260812 19:08:13.361392 29451 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.153s user 0.006s sys 0.037s
W20260812 19:08:13.361871 29451 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.153s user 0.006s sys 0.037s
I20260812 19:08:13.362169 2756 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.362656 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2263) took 2156 ms. Trace:
I20260812 19:08:13.363562 2734 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:13.364113 2744 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.363144 2756 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.365170 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.365806 29456 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:08:13.366307 2744 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.366923 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.367468 29456 raft_consensus.cc:3037] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.367784 2756 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.363430 2741 raft_consensus.cc:482] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.368945 2734 raft_consensus.cc:482] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.356993 29453 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:13.369697 2734 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.369521 2741 raft_consensus.cc:504] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.355350 2368 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.371519 2368 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:13.372407 2368 raft_consensus.cc:686] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:13.372769 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.373196 2741 leader_election.cc:288] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.320847 2623 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.374233 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.374963 29446 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:13.376147 2284 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.376927 2623 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.362218 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.378409 29459 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.329422 2707 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.379484 2465 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.353324 2400 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.363034 29451 rpcz_store.cc:276] 0812 19:08:11.206417 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.206970 (+ 553us) service_pool.cc:227] Handling call
0812 19:08:11.207311 (+ 341us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:11.207678 (+ 367us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.207705 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:11.208125 (+ 420us) log.cc:842] Serialized 26 byte log entry
0812 19:08:13.362288 (+2154163us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:13.362326 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.362396 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.362437 (+ 41us) raft_consensus.cc:1671] finished
0812 19:08:13.362449 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.362578 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1537,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":2153784,"threads_started":1,"wal-append.queue_time_us":221854,"wal-append.run_cpu_time_us":1607,"wal-append.run_wall_time_us":1001640}
I20260812 19:08:13.382357 29528 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.345168 29457 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:13.364331 2405 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.350657 2740 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.387753 29453 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:08:13.368664 1722 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.403376 2747 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.403662 2400 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.406776 2742 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.412604 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:13.413722 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.407675 2740 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.416592 29528 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.426415 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.427248 29453 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:08:13.429391 2400 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.409622 2465 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:13.408740 2623 consensus_queue.cc:1035] T 4737efcfbbb9484781f7335f47b8c2ae P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260812 19:08:13.439644 29527 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.270s user 0.017s sys 0.006s
W20260812 19:08:13.439990 29229 debug-util.cc:398] Leaking SignalData structure 0x7b0800163b20 after lost signal to thread 29179
W20260812 19:08:13.415457 2703 consensus_meta.cc:287] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 2.203s user 0.019s sys 0.026s
I20260812 19:08:13.442384 29458 raft_consensus.cc:2379] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:13.368657 2704 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:13.451831 29527 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.282s user 0.018s sys 0.006s
I20260812 19:08:13.387369 2769 consensus_queue.cc:1035] T c89f8a2f65164cd08d1e760f9005e496 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260812 19:08:13.453356 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.365826 2757 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.357254 2772 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.380218 2707 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.455708 2703 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.458902 2747 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.449636 2645 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:13.448850 2465 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.459961 2625 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.454957 2489 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.419281 2648 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:13.443037 2252 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:13.415414 2405 consensus_queue.cc:1035] T 8b7629c8adbd447596c2e821c5ac0a99 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260812 19:08:13.416019 2284 consensus_queue.cc:1035] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260812 19:08:13.461453 2757 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.426702 2368 consensus_queue.cc:237] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.483271 2757 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.476540 2252 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.486814 2252 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:13.487388 2284 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:13.488029 2284 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.488653 2284 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.489643 29456 raft_consensus.cc:2454] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:13.448130 2752 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.490317 2489 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.490661 2752 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.463840 2740 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:13.472564 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 112920us
I20260812 19:08:13.479584 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:13.480111 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 99583.3us
I20260812 19:08:13.474416 2645 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.476780 2465 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:13.499914 30625 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.501166 30625 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.475492 2648 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.504797 30625 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:13.509436 29399 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:13.511276 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.511989 29453 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:13.514143 29399 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:13.518659 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.519438 29457 raft_consensus.cc:2379] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:13.519142 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.520964 29456 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260812 19:08:13.522694 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.524042 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.515872 1722 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:13.515786 2704 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260812 19:08:13.526069 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.526162 1722 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.526531 29459 raft_consensus.cc:3032] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:13.526911 29459 raft_consensus.cc:729] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:13.527113 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.515981 2400 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:13.527859 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.477290 2647 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.522296 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.528436 29446 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:13.528491 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.526731 29579 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:13.529207 29450 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:13.524621 29453 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 3.
I20260812 19:08:13.532637 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.500218 2489 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.534268 30685 raft_consensus.cc:2379] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:13.502575 2645 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:13.533036 29453 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:13.531111 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.508841 2625 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:13.509093 2648 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.537238 30671 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:13.537633 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.538374 29569 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:13.534678 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.516043 2601 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:13.542426 30684 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:13.529142 29459 consensus_queue.cc:260] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.543864 29459 raft_consensus.cc:3037] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:13.540766 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.510059 2752 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.542361 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.543156 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.548159 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.548820 29452 raft_consensus.cc:2354] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:13.549013 29446 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:13.528267 2780 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.544710 29453 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:13.544811 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.545292 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.545730 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.522794 2740 raft_consensus.cc:2726] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:13.546630 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.534240 1722 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:13.552567 30685 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.551489 29579 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:13.551489 29456 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:13.552522 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.546811 2772 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.551966 29450 raft_consensus.cc:2396] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260812 19:08:13.542001 2489 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:13.537024 2623 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.523456 2650 consensus_queue.cc:1046] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260812 19:08:13.557686 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.559429 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.565203 2650 raft_consensus.cc:3032] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:13.556474 2489 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.565454 29578 raft_consensus.cc:2379] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:13.567261 2650 raft_consensus.cc:729] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:13.567970 2489 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:13.569022 2650 consensus_queue.cc:260] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.533800 2774 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260812 19:08:13.571074 2650 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:13.544227 2757 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.553352 2752 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.552260 2625 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:13.583043 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.586678 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:13.589246 30685 raft_consensus.cc:2379] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:08:13.590952 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 88497.6us
I20260812 19:08:13.591742 29528 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.598238 30671 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 19:08:13.600112 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.582346 2785 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.600032 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:13.599368 29528 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.554848 2780 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.612118 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.587920 2707 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.577279 2623 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.612926 2703 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.652302 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:13.617035 29399 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.68:41925 timed out after 2.798s (SENT) [suppressed 19 similar messages]
I20260812 19:08:13.659922 29527 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.660857 29527 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:13.672776 30623 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.674682 30623 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.675786 30671 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260812 19:08:13.678966 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.682650 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.685303 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.686707 30623 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.688814 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.690652 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.594547 2752 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.599496 2252 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.643755 2612 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.687990 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.675040 2757 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:13.681041 2688 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.682130 2707 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:13.703938 2657 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:13.704754 2660 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.706050 2647 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.705850 2778 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260812 19:08:13.707195 2791 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:13.709151 2776 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.155s user 0.000s sys 0.001s
I20260812 19:08:13.710428 2612 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.698243 2625 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.716315 2623 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.716835 2789 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.696303 2734 raft_consensus.cc:504] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.686336 2703 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:13.718509 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.837656 29569 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:13.717901 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.718228 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:13.724126 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 106734us
I20260812 19:08:13.747277 2368 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.610250 2785 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.694228 2769 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.786265 29574 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:13.698523 2782 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.699278 2786 raft_consensus.cc:2781] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:13.714516 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 966) took 2645 ms. Trace:
I20260812 19:08:13.702653 2757 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.785374 2780 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.867990 2780 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:13.855413 30674 rpcz_store.cc:276] 0812 19:08:11.069124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.070985 (+ 1861us) service_pool.cc:227] Handling call
0812 19:08:11.105318 (+ 34333us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:11.130235 (+ 24917us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.130272 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:11.224970 (+ 94698us) log.cc:842] Serialized 26 byte log entry
0812 19:08:13.678091 (+2453121us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:13.684417 (+ 6326us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:13.694032 (+ 9615us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:13.714063 (+ 20031us) raft_consensus.cc:1671] finished
0812 19:08:13.714084 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:13.714356 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":631,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":152718,"threads_started":1,"wal-append.queue_time_us":2413901}
I20260812 19:08:13.869041 2757 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:13.870990 2612 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.871879 2612 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.872215 2734 leader_election.cc:288] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.873201 2734 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:13.872779 2612 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.832264 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.880898 29326 raft_consensus.cc:2396] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260812 19:08:13.880743 2284 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.868677 2780 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:13.833209 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 155855us
W20260812 19:08:13.840190 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 199259us
I20260812 19:08:13.838660 29579 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:13.841367 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.792670 2489 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:13.844281 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 227280us
I20260812 19:08:13.844487 2774 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:13.803597 2610 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.803300 2687 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.807215 2252 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.852806 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.908494 29569 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:13.804842 2637 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.853607 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.842259 2789 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.864179 2785 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.793344 1722 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.791007 2688 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.785439 2707 raft_consensus.cc:482] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.841394 2703 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.904345 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.791132 2660 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:13.910251 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.910424 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.911466 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.922528 29459 raft_consensus.cc:2396] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260812 19:08:13.904541 2776 log.cc:924] Time spent T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.059s user 0.001s sys 0.000s
I20260812 19:08:13.789458 2657 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.917553 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.917614 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:13.918170 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.873803 2734 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:13.922936 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:13.912154 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.932883 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.933144 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.933624 29456 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:13.924026 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.924692 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.906188 2786 raft_consensus.cc:686] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:13.828634 2791 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.916174 2637 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.948740 2791 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.949159 2637 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:13.949749 29575 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.729s user 0.002s sys 0.033s
W20260812 19:08:13.950531 2253 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.289s user 0.035s sys 0.018s
I20260812 19:08:13.852881 2769 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.931592 29453 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:13.931934 30685 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 4.
I20260812 19:08:13.931592 29335 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:13.932449 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:13.894542 2780 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:13.934796 29528 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.922685 2707 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:13.933226 29451 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.922335 2703 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:13.933632 29569 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:13.934746 29332 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:13.935343 29579 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:13.936971 30684 raft_consensus.cc:2354] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:13.934615 29446 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:13.849686 2368 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.941301 2817 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:13.977706 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.977993 30623 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.945050 2660 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.946547 2612 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.946872 1722 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:13.905887 2774 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:13.963835 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.938087 2734 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:13.964723 2778 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:13.965471 2688 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:13.969774 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:13.975019 29528 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:13.856482 2782 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:13.943416 2789 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:13.976827 29335 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:13.985706 2804 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260812 19:08:14.020288 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 45365.7us
I20260812 19:08:13.992308 2769 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:13.993260 29575 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.772s user 0.002s sys 0.036s
I20260812 19:08:14.024001 29528 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:13.995195 2774 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:13.996821 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:13.997798 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.038941 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:13.994684 2752 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:14.039839 29580 raft_consensus.cc:3037] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:13.994632 29451 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:13.986285 29452 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 11.
I20260812 19:08:14.011044 2805 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:14.011564 2660 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:14.012436 29453 raft_consensus.cc:2454] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:08:13.925269 2801 consensus_queue.cc:1035] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260812 19:08:13.996959 2778 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:13.992102 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.051417 29456 raft_consensus.cc:2354] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:13.959784 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.054207 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:13.913371 2610 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:14.037047 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 316638us
I20260812 19:08:14.062531 29527 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:13.919567 2687 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.063803 2610 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.039614 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 883) took 2926 ms. Trace:
I20260812 19:08:14.066120 29575 rpcz_store.cc:276] 0812 19:08:11.098446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.103301 (+ 4855us) service_pool.cc:227] Handling call
0812 19:08:11.105014 (+ 1713us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:11.105896 (+ 882us) raft_consensus.cc:1520] Early marking committed up to index 10
0812 19:08:11.191270 (+ 85374us) log.cc:842] Serialized 14 byte log entry
0812 19:08:11.219656 (+ 28386us) log.cc:842] Serialized 14 byte log entry
0812 19:08:13.122174 (+1902518us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b48002301d8. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6ecb57f1 00007f9c6ecac8e7 00007f9c6ecac77a 00007f9c6eda095f 00007f9c6eda0627 00007f9c6edb4b2c 00007f9c6edb4ad8 00007f9c6edb4a58 00007f9c6edb4a13 00007f9c6edb3cbc 00007f9c6dd35be6
0812 19:08:14.024047 (+901873us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:14.024101 (+ 54us) raft_consensus.cc:1631] Marking committed up to 10
0812 19:08:14.024147 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.024227 (+ 80us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.024336 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1849,"prepare.queue_time_us":95112,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":101,"raft.queue_time_us":87278,"raft.run_cpu_time_us":27101,"raft.run_wall_time_us":1880550,"spinlock_wait_cycles":3279104,"thread_start_us":2803064,"threads_started":1}
I20260812 19:08:14.068274 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:14.068722 29580 raft_consensus.cc:2454] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:14.069068 29575 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:14.042344 29528 leader_election.cc:302] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.039727 29569 raft_consensus.cc:3037] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:14.070818 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed9c55b662464bbd8e692e47a2f7cc2c" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.070941 2489 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.071296 30623 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.072669 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.072943 2777 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:14.073304 29580 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:14.073614 2777 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.074323 2777 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:13.986616 2817 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.039192 29579 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:14.078794 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:14.078887 2752 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:14.022921 2804 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.019717 2688 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:14.080255 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.080850 2688 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.082267 2807 raft_consensus.cc:2781] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:14.083209 2807 raft_consensus.cc:686] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:14.048627 30671 raft_consensus.cc:1261] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260812 19:08:14.022945 2822 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:14.050307 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:14.085367 2822 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:14.086052 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.086186 2822 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.086545 2657 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.054508 2778 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:14.087571 29527 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:14.085012 2807 consensus_queue.cc:237] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.039532 2789 raft_consensus.cc:2726] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:14.025990 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.092276 29445 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:14.055267 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.093374 30686 raft_consensus.cc:2454] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
W20260812 19:08:14.048880 2253 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.387s user 0.036s sys 0.019s
I20260812 19:08:14.053831 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.094831 30681 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:14.095664 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.096341 30670 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:14.094197 29527 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.097365 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.094517 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.081153 29528 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.098028 30670 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:14.099339 29528 leader_election.cc:398] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:14.100268 29335 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:14.100842 2789 raft_consensus.cc:2781] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:14.101771 2789 raft_consensus.cc:686] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:14.102821 2804 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:14.103569 2789 consensus_queue.cc:237] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.049815 2660 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.105168 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:14.105765 29445 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.106428 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.049485 2805 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.113283 29445 raft_consensus.cc:2454] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
W20260812 19:08:14.113559 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080003d220 after lost signal to thread 29236
I20260812 19:08:14.113933 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.114530 30670 raft_consensus.cc:2454] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:14.115347 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.115878 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.116060 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.116537 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.116711 30670 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 4.
I20260812 19:08:14.117482 29452 raft_consensus.cc:2396] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260812 19:08:14.117847 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.118227 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:14.118415 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:14.046998 2769 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:14.118759 29451 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:14.118939 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.114449 29446 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:08:14.119709 29445 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 4.
I20260812 19:08:14.044652 2786 consensus_queue.cc:237] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.036844 2649 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:14.137681 2649 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.021174 2637 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:14.108196 2660 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:14.120484 29457 raft_consensus.cc:1261] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260812 19:08:14.120879 29456 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:14.144174 29451 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:14.144428 29580 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:14.123207 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.080281 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.090888 2489 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 19:08:14.060366 2601 consensus_meta.cc:287] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.511s user 0.010s sys 0.000s
I20260812 19:08:14.158602 2601 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:14.120507 29326 raft_consensus.cc:2396] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260812 19:08:14.088722 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:14.159473 2805 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.159824 29575 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:14.097782 29569 raft_consensus.cc:2454] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:14.162295 30670 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 10.
I20260812 19:08:14.132105 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.146051 29579 raft_consensus.cc:2379] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:14.146828 2801 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:14.180981 29580 raft_consensus.cc:1226] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:08:14.182137 29569 raft_consensus.cc:1261] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260812 19:08:14.182314 29282 connection.cc:232] RPC call timeout handler was delayed by -2.3623566350433975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260812 19:08:14.155315 2645 consensus_meta.cc:287] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.611s user 0.010s sys 0.003s
I20260812 19:08:14.164037 30681 raft_consensus.cc:2454] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
W20260812 19:08:14.139642 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 45823us
I20260812 19:08:14.142509 2769 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.184437 29399 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:14.081110 2284 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.086218 2785 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260812 19:08:14.183511 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 60426.5us
I20260812 19:08:14.071319 2687 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:14.088375 2368 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:14.085013 2688 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:14.204568 29334 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 3.072s user 0.006s sys 0.021s
I20260812 19:08:14.181094 2801 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.205192 2801 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:14.158942 2822 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:14.206652 2648 consensus_meta.cc:287] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.656s user 0.010s sys 0.004s
I20260812 19:08:14.207664 2648 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.209156 2822 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:14.080184 2817 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:14.210093 2782 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.211176 2817 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:14.212041 2817 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.209491 29575 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260812 19:08:14.197026 29528 leader_election.cc:302] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.198002 2647 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:14.238570 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:14.239099 29334 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.106s user 0.006s sys 0.021s
I20260812 19:08:14.240044 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 931) took 3137 ms. Trace:
I20260812 19:08:14.213881 2752 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:14.148322 2637 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.535388 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:14.544781 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:14.477715 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 465878us
I20260812 19:08:14.355232 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:14.426908 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.540714 29341 raft_consensus.cc:3037] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.376178 29337 raft_consensus.cc:1226] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260812 19:08:14.545109 2742 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:14.354435 2648 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:14.503646 2465 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.983s user 0.012s sys 0.003s
I20260812 19:08:14.302094 2822 raft_consensus.cc:2781] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:14.471567 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.361352 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.546718 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.266685 29456 raft_consensus.cc:2454] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:08:14.442198 2757 consensus_meta.cc:287] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.572s user 0.003s sys 0.012s
I20260812 19:08:14.550207 29446 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:14.550115 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:14.243116 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:14.545622 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 330029us
I20260812 19:08:14.591519 29445 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.547462 2752 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:14.540752 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:14.556331 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.558605 29399 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.298421 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 972) took 3114 ms. Trace:
I20260812 19:08:14.216387 2805 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:14.198940 2785 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.346222 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b7629c8adbd447596c2e821c5ac0a99" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.240384 29334 rpcz_store.cc:276] 0812 19:08:11.102292 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.102428 (+ 136us) service_pool.cc:227] Handling call
0812 19:08:11.113872 (+ 11444us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:11.114238 (+ 366us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.114269 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:11.123643 (+ 9374us) log.cc:842] Serialized 26 byte log entry
0812 19:08:14.239703 (+3116060us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:14.239743 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.239811 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:14.239854 (+ 43us) raft_consensus.cc:1671] finished
0812 19:08:14.239867 (+ 13us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.239989 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":564,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":1918,"thread_start_us":3106611,"threads_started":1,"wal-append.queue_time_us":290070,"wal-append.run_cpu_time_us":1946,"wal-append.run_wall_time_us":1007678}
I20260812 19:08:14.543022 30623 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:14.565867 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080036e820 after lost signal to thread 29379
W20260812 19:08:14.597103 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 54186.2us
I20260812 19:08:14.594918 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:14.595317 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:14.603441 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44917.2us
I20260812 19:08:14.604090 29399 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.604525 30676 rpcz_store.cc:276] 0812 19:08:11.184225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.225119 (+ 40894us) service_pool.cc:227] Handling call
0812 19:08:11.225426 (+ 307us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:11.260396 (+ 34970us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:11.260426 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:11.260476 (+ 50us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:11.260507 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.298144 (+3037637us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b5400764dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:14.298197 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.298349 (+ 152us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":80013184}
I20260812 19:08:14.594899 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:14.566530 29578 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 887) took 3419 ms (client timeout 2159 ms). Trace:
I20260812 19:08:14.580571 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:14.577298 29281 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.241617 30671 raft_consensus.cc:2454] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:14.568528 30629 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:14.588191 29452 raft_consensus.cc:2396] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260812 19:08:14.591909 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:14.608668 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 1013) took 1022 ms. Trace:
I20260812 19:08:14.548308 2071 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:14.609369 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0fa0 after lost signal to thread 29496
I20260812 19:08:14.610262 29579 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:08:14.615252 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080031d760 after lost signal to thread 29500
I20260812 19:08:14.610903 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:14.607057 29578 rpcz_store.cc:269] 0812 19:08:11.120243 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:11.121694 (+ 1451us) service_pool.cc:227] Handling call
0812 19:08:14.539256 (+3417562us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b5400136354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:14.539415 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5569772288}
W20260812 19:08:14.616763 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08003fa5a0 after lost signal to thread 29622
W20260812 19:08:14.617069 29229 debug-util.cc:398] Leaking SignalData structure 0x7b0800143260 after lost signal to thread 29623
I20260812 19:08:14.615000 30684 rpcz_store.cc:276] 0812 19:08:13.586547 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.586752 (+ 205us) service_pool.cc:227] Handling call
0812 19:08:14.608484 (+1021732us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b54006a2054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:14.608611 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":747916032}
I20260812 19:08:14.613008 30623 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.618844 30623 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:14.605667 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:14.623378 29229 debug-util.cc:398] Leaking SignalData structure 0x7b0800379160 after lost signal to thread 30721
W20260812 19:08:14.623802 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08002c69e0 after lost signal to thread 30724
W20260812 19:08:14.624223 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08003caae0 after lost signal to thread 2125
W20260812 19:08:14.624912 29229 debug-util.cc:398] Leaking SignalData structure 0x7b080038dc20 after lost signal to thread 2575
W20260812 19:08:14.625221 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08003d9420 after lost signal to thread 2579
W20260812 19:08:14.625772 29229 debug-util.cc:398] Leaking SignalData structure 0x7b08003bede0 after lost signal to thread 2629
I20260812 19:08:14.242267 2660 raft_consensus.cc:482] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:14.613468 29445 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
W20260812 19:08:14.651664 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 83204us
W20260812 19:08:14.652587 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 44685us
I20260812 19:08:14.199584 2645 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.693267 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da12498df45141cd80e5cb36e68775c5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.694028 30676 raft_consensus.cc:3037] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.692936 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2743b6275ccc4fbdbe5f71c667adee8b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.697645 29580 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2254) took 1041 ms. Trace:
I20260812 19:08:14.698097 29580 rpcz_store.cc:276] 0812 19:08:13.656554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.145218 (+488664us) service_pool.cc:227] Handling call
0812 19:08:14.145571 (+ 353us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.195430 (+ 49859us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.697336 (+501906us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b5400177354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:14.697582 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44101504}
I20260812 19:08:14.694106 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:14.700160 29446 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:14.692198 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 1003) took 1183 ms. Trace:
I20260812 19:08:14.709379 29341 raft_consensus.cc:1261] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260812 19:08:14.703660 30679 rpcz_store.cc:276] 0812 19:08:13.515445 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.361080 (+845635us) service_pool.cc:227] Handling call
0812 19:08:14.692115 (+331035us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:14.581275 2687 raft_consensus.cc:482] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:14.718587 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64325c7581194fafac6e6e2b485fbdc2" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:14.654312 2596 consensus_meta.cc:287] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 3.745s user 0.025s sys 0.006s
I20260812 19:08:14.736286 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.737884 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 920) took 4221 ms. Trace:
I20260812 19:08:14.736110 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:14.739122 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.596158 2752 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:14.740474 29341 rpcz_store.cc:276] 0812 19:08:10.489148 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:10.489296 (+ 148us) service_pool.cc:227] Handling call
0812 19:08:10.489573 (+ 277us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:10.913463 (+423890us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400279d20. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d9e730 00007f9c69e10300 00007f9c69e11371 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc
0812 19:08:14.540261 (+3626798us) spinlock_profiling.cc:243] Waited 3.53 s on lock 0x7b58007274b0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d89f63 00007f9c69dedaba 00007f9c69e053af 00007f9c69e06b4e 00007f9c69e10c89 00007f9c69e11556 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684
0812 19:08:14.709866 (+169605us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.710239 (+ 373us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8444142336}
W20260812 19:08:14.741329 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55566.1us
I20260812 19:08:14.724000 30676 raft_consensus.cc:2454] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:14.609376 2822 raft_consensus.cc:686] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:14.775542 2596 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.782279 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015555682f5f4cf293ca3718be163873" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:14.704623 2660 raft_consensus.cc:504] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.692389 2647 raft_consensus.cc:2741] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260812 19:08:14.692109 2071 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.735878 2706 consensus_queue.cc:1035] T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260812 19:08:14.717742 2791 raft_consensus.cc:2781] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:14.793514 30685 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260812 19:08:14.719676 2400 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:14.743582 2687 raft_consensus.cc:504] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.704169 2625 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:14.608270 2785 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 11 FOLLOWER]: Advancing to term 12
I20260812 19:08:14.550870 2777 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.357146 2703 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.840994 30674 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:14.842120 2703 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:14.844870 2687 leader_election.cc:288] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:14.847071 2777 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:14.739699 2379 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260812 19:08:14.650223 2742 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:14.653856 2772 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
W20260812 19:08:14.808100 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 104836us
I20260812 19:08:14.706045 2405 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:14.794265 2829 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:14.586714 2465 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.719480 2648 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:14.823088 30629 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:14.823691 29567 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 890) took 1642 ms. Trace:
I20260812 19:08:14.649865 2805 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.795151 2817 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:14.651018 2757 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.832948 2625 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:14.840097 2828 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:14.821244 2400 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.859288 2742 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:14.917011 30676 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260812 19:08:14.836939 2791 raft_consensus.cc:686] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:14.856930 2377 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:14.916608 29567 rpcz_store.cc:276] 0812 19:08:13.181360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.181509 (+ 149us) service_pool.cc:227] Handling call
0812 19:08:13.181862 (+ 353us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:13.202946 (+ 21084us) raft_consensus.cc:1520] Early marking committed up to index 7
0812 19:08:13.203244 (+ 298us) log.cc:842] Serialized 14 byte log entry
0812 19:08:13.203416 (+ 172us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:13.203462 (+ 46us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:13.203508 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.823450 (+1619942us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b540012be54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:14.823507 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.823634 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":158613,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":80,"raft.queue_time_us":315,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"spinlock_wait_cycles":47991040}
W20260812 19:08:14.861943 2801 consensus_meta.cc:287] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.656s user 0.012s sys 0.000s
I20260812 19:08:14.918282 2772 raft_consensus.cc:686] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:14.932700 29572 raft_consensus.cc:2379] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:14.934473 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2573) took 1023 ms. Trace:
W20260812 19:08:14.935591 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 112597us
I20260812 19:08:14.930601 2400 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
W20260812 19:08:14.937969 2778 consensus_meta.cc:287] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.849s user 0.007s sys 0.007s
I20260812 19:08:14.938365 2778 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:14.917615 2829 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:14.934912 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 357700us
W20260812 19:08:14.954113 29576 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.029s sys 0.079s
I20260812 19:08:14.923911 2625 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:08:14.978961 29576 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s user 0.029s sys 0.079s
I20260812 19:08:14.990290 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:14.991118 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:14.940322 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34302 (request call id 1019) took 1045 ms. Trace:
I20260812 19:08:14.934239 2742 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:14.936232 29450 rpcz_store.cc:276] 0812 19:08:13.911369 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.911474 (+ 105us) service_pool.cc:227] Handling call
0812 19:08:13.911820 (+ 346us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:13.915304 (+ 3484us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:13.915325 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:13.937015 (+ 21690us) log.cc:842] Serialized 26 byte log entry
0812 19:08:14.129399 (+192384us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:14.129442 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.129515 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:14.934219 (+804704us) raft_consensus.cc:1671] finished
0812 19:08:14.934241 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.934397 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3089,"raft.queue_time_us":3202,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7406,"thread_start_us":154273,"threads_started":1,"wal-append.queue_time_us":867345}
I20260812 19:08:14.945786 29567 raft_consensus.cc:2932] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, a34c47dd29c14748a018d716ac6c39df (127.26.205.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } } }
I20260812 19:08:14.990384 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2344) took 1844 ms. Trace:
I20260812 19:08:15.003919 29576 rpcz_store.cc:276] 0812 19:08:13.146391 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.433243 (+1286852us) service_pool.cc:227] Handling call
0812 19:08:14.433646 (+ 403us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.443548 (+ 9902us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:14.443580 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:14.444073 (+ 493us) log.cc:842] Serialized 26 byte log entry
0812 19:08:14.990049 (+545976us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:14.990091 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.990157 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:14.990204 (+ 47us) raft_consensus.cc:1671] finished
0812 19:08:14.990215 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:14.990326 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":10478,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":10048,"thread_start_us":545649,"threads_started":1,"wal-append.queue_time_us":495390}
I20260812 19:08:14.919047 2777 consensus_queue.cc:1046] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.075s
I20260812 19:08:15.041330 29580 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:15.037463 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.043608 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:15.044260 29326 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:14.940364 2778 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:15.032604 1362 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s user 0.031s sys 0.077s
I20260812 19:08:14.935609 2451 raft_consensus.cc:2726] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260812 19:08:15.038887 2734 consensus_meta.cc:287] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.017s user 0.008s sys 0.004s
W20260812 19:08:15.047261 2707 consensus_meta.cc:287] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.064s user 0.013s sys 0.001s
I20260812 19:08:15.070897 29326 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
W20260812 19:08:15.067875 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 348050us
I20260812 19:08:14.946357 2377 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:14.865176 2822 consensus_queue.cc:237] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.077282 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:14.924881 2648 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.079906 30629 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:08:15.060719 2774 consensus_meta.cc:287] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.013s user 0.006s sys 0.007s
I20260812 19:08:14.992518 2601 consensus_queue.cc:1035] T 14e182b70d344161a275142a344cd1ba P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260812 19:08:15.090617 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 498715us
W20260812 19:08:15.086308 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 410538us
I20260812 19:08:14.933056 2791 consensus_queue.cc:237] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.068732 2645 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:14.946547 2801 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.098327 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.099733 29580 raft_consensus.cc:2454] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
W20260812 19:08:15.098613 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 487758us
W20260812 19:08:15.083344 2835 log.cc:924] Time spent T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.166s user 0.000s sys 0.001s
I20260812 19:08:15.103574 29399 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:15.092105 2657 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s user 0.003s sys 0.013s
I20260812 19:08:15.002439 30674 rpcz_store.cc:276] 0812 19:08:13.894813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.899522 (+ 4709us) service_pool.cc:227] Handling call
0812 19:08:14.940115 (+1040593us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54005eecd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:14.940263 (+ 148us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1770944256}
I20260812 19:08:15.104544 2071 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.091068 2596 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.117864 29577 raft_consensus.cc:2354] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:08:15.111183 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:15.094910 2777 raft_consensus.cc:2781] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:15.086015 2828 raft_consensus.cc:686] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:15.078231 2734 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.084695 2707 raft_consensus.cc:504] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.122519 29446 raft_consensus.cc:2454] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:15.139775 29575 raft_consensus.cc:1261] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260812 19:08:15.131821 30671 raft_consensus.cc:1261] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260812 19:08:15.118507 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.077353 2772 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.158254 2757 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.092790 2774 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.200356 29527 leader_election.cc:398] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:15.200699 29527 leader_election.cc:401] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260812 19:08:15.200865 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.173776 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.174890 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.201819 29569 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 9
W20260812 19:08:15.119891 2657 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s user 0.003s sys 0.013s
I20260812 19:08:15.209082 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:15.216284 2861 raft_consensus.cc:2781] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election won for term 7
W20260812 19:08:15.199618 2838 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.244s user 0.000s sys 0.001s
I20260812 19:08:15.220400 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2578) took 1301 ms. Trace:
I20260812 19:08:15.165009 2829 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:15.221087 29459 rpcz_store.cc:276] 0812 19:08:13.919107 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.923170 (+ 4063us) service_pool.cc:227] Handling call
0812 19:08:13.938041 (+ 14871us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:13.938405 (+ 364us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:13.938434 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:13.939247 (+ 813us) log.cc:842] Serialized 26 byte log entry
0812 19:08:14.376977 (+437730us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:14.377022 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:14.377094 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:15.220168 (+843074us) raft_consensus.cc:1671] finished
0812 19:08:15.220190 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:15.220337 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2278,"raft.queue_time_us":19137,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":13317,"thread_start_us":366769,"threads_started":1,"wal-append.queue_time_us":864931}
I20260812 19:08:15.226238 29569 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9.
I20260812 19:08:15.227576 30623 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.228446 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:15.229319 30671 raft_consensus.cc:2354] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:15.231809 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.162686 2801 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.236676 29527 leader_election.cc:398] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:15.253301 29527 leader_election.cc:401] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260812 19:08:15.176987 2780 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 1.212s user 0.011s sys 0.003s
I20260812 19:08:15.237438 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.217536 2734 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.239722 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:15.242122 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.252557 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.254055 30686 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:15.165841 2465 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:15.263049 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.266404 29580 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:15.267359 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.267446 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.263657 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.268863 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.269376 30675 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:15.268779 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.271549 30623 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.272284 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.273064 29575 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:15.269486 29580 raft_consensus.cc:3037] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:15.273872 29569 raft_consensus.cc:1261] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260812 19:08:15.176873 2777 raft_consensus.cc:686] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:15.283143 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 184781us
I20260812 19:08:15.224530 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.200040 2805 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.235419 2645 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6.
I20260812 19:08:15.292238 29580 raft_consensus.cc:2454] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:15.297083 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:15.296119 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 197738us
W20260812 19:08:15.301641 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 190540us
I20260812 19:08:15.339335 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:15.301151 29574 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57278 (request call id 2247) took 1648 ms. Trace:
I20260812 19:08:15.284437 2782 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:15.334632 29580 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260812 19:08:15.259230 2596 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:15.336540 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.342308 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.342624 30675 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:08:15.303238 2861 raft_consensus.cc:686] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:15.344247 30675 raft_consensus.cc:1261] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260812 19:08:15.333906 2855 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.110s user 0.000s sys 0.001s
I20260812 19:08:15.287784 2870 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260812 19:08:15.340245 29574 rpcz_store.cc:276] 0812 19:08:13.652680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.117814 (+465134us) service_pool.cc:227] Handling call
0812 19:08:14.118261 (+ 447us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.118631 (+ 370us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:14.118659 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:14.564660 (+446001us) log.cc:842] Serialized 26 byte log entry
0812 19:08:15.094670 (+530010us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:15.295494 (+200824us) log.cc:842] Serialized 14 byte log entry
0812 19:08:15.296891 (+ 1397us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:15.296984 (+ 93us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:15.300927 (+ 3943us) raft_consensus.cc:1671] finished
0812 19:08:15.300949 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:15.301087 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40042,"raft.queue_time_us":18584,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":15679,"thread_start_us":589002,"threads_started":2,"wal-append.queue_time_us":630673}
I20260812 19:08:15.363831 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.355695 2710 consensus_queue.cc:1035] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260812 19:08:15.360025 29579 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:15.354833 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:15.285796 2780 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.372468 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:15.379316 29282 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:15.390192 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:08:15.405012 29572 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2243) took 1753 ms (client timeout 1605 ms). Trace:
W20260812 19:08:15.405517 29572 rpcz_store.cc:269] 0812 19:08:13.651550 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:13.651669 (+ 119us) service_pool.cc:227] Handling call
0812 19:08:15.404757 (+1753088us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540009e554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:15.404929 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2871697664}
I20260812 19:08:15.410300 29578 raft_consensus.cc:1226] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:15.384807 2610 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:15.390311 2860 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:15.393358 1986 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:15.378187 2132 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260812 19:08:15.336436 2828 consensus_queue.cc:237] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:15.414143 2785 consensus_meta.cc:287] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.571s user 0.008s sys 0.005s
I20260812 19:08:15.408627 2648 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.383932 2219 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260812 19:08:15.436578 2877 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260812 19:08:15.413075 2774 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.425005 1986 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:15.420181 2610 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:15.429883 2805 raft_consensus.cc:2781] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260812 19:08:15.453498 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 114218us
I20260812 19:08:15.378645 2777 consensus_queue.cc:237] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.422910 2892 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:15.418356 2195 consensus_queue.cc:1035] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260812 19:08:15.454586 2400 consensus_meta.cc:287] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.517s user 0.004s sys 0.007s
I20260812 19:08:15.455229 1986 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:15.472193 2780 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.454634 2610 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:15.380537 2861 consensus_queue.cc:237] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.459964 2805 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:15.444854 2785 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.461158 2892 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:15.449684 2617 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260812 19:08:15.502214 2892 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:08:15.504281 2867 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.091s user 0.000s sys 0.001s
I20260812 19:08:15.506037 2908 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:15.506807 2908 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:15.507537 2908 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:15.493062 2805 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
W20260812 19:08:15.509202 2890 log.cc:924] Time spent T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
W20260812 19:08:15.511704 29623 heartbeater.cc:644] Failed to heartbeat to 127.26.205.126:35929 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.26.205.126:35929 timed out after 15.000s (SENT)
I20260812 19:08:15.512874 2891 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:15.513692 2891 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.524405 2785 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 12 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
W20260812 19:08:15.525272 2752 consensus_meta.cc:287] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.711s user 0.010s sys 0.002s
I20260812 19:08:15.526450 29574 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 12 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 11. Current term is 12. Ops: []
I20260812 19:08:15.478983 2400 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.526865 29569 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:15.531663 29569 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:15.533121 29567 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:15.553987 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.555382 29569 raft_consensus.cc:1226] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
W20260812 19:08:15.558651 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 209932us
I20260812 19:08:15.563678 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.564695 2769 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:15.530136 2742 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.571976 2742 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.575189 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:15.576558 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.576558 2905 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:15.515324 2625 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.578358 2905 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.580569 2625 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.581542 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 977) took 1067 ms. Trace:
I20260812 19:08:15.604031 30676 rpcz_store.cc:276] 0812 19:08:14.514061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.778964 (+264903us) service_pool.cc:227] Handling call
0812 19:08:14.793065 (+ 14101us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.927491 (+134426us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:15.581276 (+653785us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b5400465854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:15.581476 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":175730048}
I20260812 19:08:15.605181 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:15.607035 30625 leader_election.cc:302] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.580607 2660 leader_election.cc:288] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:15.584656 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57270 (request call id 2346) took 2437 ms (client timeout 1532 ms). Trace:
I20260812 19:08:15.585875 2905 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.587785 2647 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:15.529503 2752 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.572115 2400 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.590907 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.621605 30674 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:15.593017 2475 raft_consensus.cc:3032] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:15.530498 2891 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.596748 2377 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.643325 29527 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:15.600456 2195 raft_consensus.cc:2781] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260812 19:08:15.610610 29577 rpcz_store.cc:269] 0812 19:08:13.147032 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.426570 (+1279538us) service_pool.cc:227] Handling call
0812 19:08:15.584449 (+1157879us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b5400177d54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:15.584583 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1272477056}
I20260812 19:08:15.623982 29281 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:15.557684 2596 raft_consensus.cc:686] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 9 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:15.597836 2869 log.cc:924] Time spent T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.172s user 0.001s sys 0.001s
I20260812 19:08:15.600836 2803 consensus_queue.cc:1046] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260812 19:08:15.640244 29575 raft_consensus.cc:1226] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:15.641745 30625 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.642972 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
W20260812 19:08:15.662446 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 201575us
I20260812 19:08:15.662953 30676 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 10.
I20260812 19:08:15.664149 30676 raft_consensus.cc:3032] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:15.648512 29281 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.664512 30676 raft_consensus.cc:729] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:15.614548 2888 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:15.641600 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:15.668339 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.651773 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.631848 2647 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:15.633879 2789 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.090s user 0.004s sys 0.011s
I20260812 19:08:15.649658 29576 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:15.641455 2905 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:15.669106 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 271893us
I20260812 19:08:15.669140 30674 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 10.
I20260812 19:08:15.677017 29282 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:15.667891 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.629712 2475 raft_consensus.cc:729] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:15.651983 2780 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:15.651101 2195 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:15.672395 29528 leader_election.cc:302] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:15.647866 2405 raft_consensus.cc:2726] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:08:15.665287 30676 consensus_queue.cc:260] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.717473 30676 raft_consensus.cc:3037] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:15.721342 2876 raft_consensus.cc:2726] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260812 19:08:15.722249 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 49942.6us
I20260812 19:08:15.723378 29528 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:15.678594 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.671721 29281 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.629765 2219 consensus_queue.cc:1035] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260812 19:08:15.724535 29528 leader_election.cc:302] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:15.710424 2780 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:15.727335 2780 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:08:15.727133 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 55601.5us
I20260812 19:08:15.728485 2822 raft_consensus.cc:2781] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:15.729219 2822 raft_consensus.cc:686] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:15.730192 2283 consensus_queue.cc:1035] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:08:15.730762 2710 consensus_queue.cc:1046] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 19:08:15.732689 2283 consensus_queue.cc:1046] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:15.733745 2611 consensus_queue.cc:1046] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260812 19:08:15.735335 2846 log.cc:924] Time spent T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.609s user 0.000s sys 0.005s
I20260812 19:08:15.734956 2874 consensus_queue.cc:1046] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:08:15.735960 2657 raft_consensus.cc:3032] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:15.736740 2803 raft_consensus.cc:3032] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:15.736523 2660 raft_consensus.cc:3032] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:15.738116 2660 raft_consensus.cc:729] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:15.726035 2648 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:15.709745 29577 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260812 19:08:15.708927 2905 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.740574 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.711237 2785 raft_consensus.cc:2781] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:15.711788 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.741534 29575 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:15.740839 2610 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:15.674608 2789 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.131s user 0.005s sys 0.011s
I20260812 19:08:15.711710 2475 consensus_queue.cc:260] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.736557 2657 raft_consensus.cc:729] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:15.726078 2195 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:15.737514 2803 raft_consensus.cc:729] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:15.746505 29567 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:15.743551 2657 consensus_queue.cc:260] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.712845 2649 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.714227 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:15.749038 29578 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:15.739187 2660 consensus_queue.cc:260] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.753072 2782 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925) with higher term. Message: Invalid argument: T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
W20260812 19:08:15.754733 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 77787.8us
I20260812 19:08:15.671139 2369 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:15.716168 2647 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.751883 2660 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
W20260812 19:08:15.726527 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 48012.5us
I20260812 19:08:15.748229 2657 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:15.758898 30676 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260812 19:08:15.761989 29575 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:15.679710 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:15.783542 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:15.667660 2804 consensus_meta.cc:287] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.564s user 0.015s sys 0.005s
I20260812 19:08:15.730115 2617 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:15.774222 29569 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9.
I20260812 19:08:15.777163 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.777405 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1026) took 1165 ms. Trace:
I20260812 19:08:15.780311 29281 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.680176 2891 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:15.801642 2822 consensus_queue.cc:237] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.802248 2891 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.803102 29337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2334) took 5231 ms. Trace:
I20260812 19:08:15.798329 2804 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:15.783689 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 104020us
I20260812 19:08:15.764380 29578 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:15.803500 29337 rpcz_store.cc:276] 0812 19:08:10.571764 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:10.571892 (+ 128us) service_pool.cc:227] Handling call
0812 19:08:14.375898 (+3804006us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.579182 (+203284us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:15.802908 (+1223726us) spinlock_profiling.cc:243] Waited 3.8 s on lock 0x7b54002323d0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:15.803044 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8743082752}
I20260812 19:08:15.743469 2475 raft_consensus.cc:3037] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:15.729172 2132 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:15.781913 2752 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:15.788857 2637 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.781116 2648 raft_consensus.cc:686] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:15.745225 2803 consensus_queue.cc:260] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:15.793634 2901 log.cc:924] Time spent T c7f7d48e270c456b9b7eeab6f3165a89 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260812 19:08:15.796811 30623 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.797283 30625 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.710712 2888 raft_consensus.cc:2726] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:15.831996 2803 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:15.832681 2769 raft_consensus.cc:2726] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 19:08:15.809767 2379 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:15.822175 2870 consensus_queue.cc:1046] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260812 19:08:15.796119 30679 rpcz_store.cc:276] 0812 19:08:14.612255 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.776719 (+164464us) service_pool.cc:227] Handling call
0812 19:08:14.793082 (+ 16363us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:14.815666 (+ 22584us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:14.815698 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:14.816315 (+ 617us) log.cc:842] Serialized 26 byte log entry
0812 19:08:15.168018 (+351703us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:15.168061 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:15.168133 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:15.777212 (+609079us) raft_consensus.cc:1671] finished
0812 19:08:15.777232 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:15.777348 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23598,"raft.queue_time_us":25671,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8964,"thread_start_us":328759,"threads_started":1,"wal-append.queue_time_us":507675}
I20260812 19:08:15.800348 2596 consensus_queue.cc:237] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.827519 2817 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260812 19:08:15.711721 2899 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:15.840582 2870 raft_consensus.cc:3032] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:15.841066 2870 raft_consensus.cc:729] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:15.841346 2892 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.841898 2870 consensus_queue.cc:260] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:15.843276 2870 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:15.780925 2283 consensus_queue.cc:1035] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260812 19:08:15.843604 2892 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.804245 2891 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.848490 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.772200 29282 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:15.803524 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:15.766259 2625 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260812 19:08:15.808310 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 53037.8us
I20260812 19:08:15.808648 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:15.810719 29281 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:15.810613 29399 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:15.857897 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 85886.8us
W20260812 19:08:15.858400 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 47896.7us
I20260812 19:08:15.822736 30670 raft_consensus.cc:2354] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5.
I20260812 19:08:15.804749 2369 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 19:08:15.810036 2132 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:15.825511 2610 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:15.825176 2648 consensus_queue.cc:237] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.829917 30625 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.896800 29575 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260812 19:08:15.896917 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 67095.9us
I20260812 19:08:15.810353 2905 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:15.838963 2601 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:15.846679 2899 raft_consensus.cc:482] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:15.680539 2828 consensus_queue.cc:1035] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.126s
I20260812 19:08:15.838938 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.905695 2707 leader_election.cc:288] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:15.855340 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:15.865444 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:08:15.888494 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 77993.7us
I20260812 19:08:15.890817 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
W20260812 19:08:15.972751 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 113477us
I20260812 19:08:15.972851 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:15.973529 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.856555 2625 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:15.865198 2369 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Starting leader election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:15.796264 2647 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:15.785687 2785 raft_consensus.cc:686] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:15.802683 2904 consensus_queue.cc:1035] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.126s
I20260812 19:08:15.902841 30685 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [term 9 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:15.903782 30675 raft_consensus.cc:1226] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:15.837443 2908 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:15.889060 2132 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:15.993810 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:15.886567 2891 raft_consensus.cc:2781] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:15.996740 2891 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:15.997176 2891 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:15.887444 2888 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:15.898837 2358 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 19:08:15.891541 2892 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:15.971207 29580 raft_consensus.cc:1226] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
W20260812 19:08:15.971210 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 63346.1us
I20260812 19:08:15.973657 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:15.897744 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:15.974112 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:16.040799 2803 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260812 19:08:16.040954 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 66909us
I20260812 19:08:15.973031 29574 raft_consensus.cc:2379] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:15.976706 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.982937 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.902361 2769 raft_consensus.cc:2781] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:16.043635 2769 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.044090 2769 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:16.044272 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.983593 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 944) took 1436 ms. Trace:
I20260812 19:08:16.045712 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.046120 30681 rpcz_store.cc:276] 0812 19:08:14.547163 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.717828 (+170665us) service_pool.cc:227] Handling call
0812 19:08:15.976058 (+1258230us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b54006a2054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:15.983534 (+ 7476us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1751496576}
I20260812 19:08:15.988691 2369 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:15.987069 2647 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.054042 2647 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:16.054741 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 157080us
I20260812 19:08:16.055823 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.056048 30625 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:16.056636 29331 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:16.057477 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:15.989053 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.986855 2625 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:15.989707 2785 consensus_queue.cc:237] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.058010 2703 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:15.989707 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.058462 2703 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.058938 29338 raft_consensus.cc:2396] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260812 19:08:16.006098 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.005034 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.010337 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2585) took 1886 ms. Trace:
I20260812 19:08:15.931043 2899 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:16.086615 2904 raft_consensus.cc:2781] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:16.060935 29458 rpcz_store.cc:276] 0812 19:08:14.124018 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.140920 (+ 16902us) service_pool.cc:227] Handling call
0812 19:08:15.993647 (+1852727us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b580016dd98. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:16.010264 (+ 16617us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17566080}
I20260812 19:08:15.983168 2905 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260812 19:08:15.987090 2929 log.cc:924] Time spent T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260812 19:08:16.058884 2703 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:16.061733 29331 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:16.008839 30674 raft_consensus.cc:1261] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260812 19:08:15.902721 2778 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.099s
I20260812 19:08:16.084362 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.153121 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:15.988055 2637 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 19:08:16.154618 2637 raft_consensus.cc:482] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:16.155016 2637 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:16.083943 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.158301 29339 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:16.159265 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.160429 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.161506 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:16.086526 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 112924us
I20260812 19:08:16.085201 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.162729 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:16.163252 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.163733 29326 raft_consensus.cc:2354] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:16.165257 29338 raft_consensus.cc:1261] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260812 19:08:16.089192 29335 raft_consensus.cc:2396] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260812 19:08:16.089525 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:16.166751 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 77322.4us
I20260812 19:08:16.167425 29339 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:16.149708 29574 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:16.171222 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:16.171947 29574 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:16.171797 2637 raft_consensus.cc:504] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:16.018198 2916 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.118s user 0.000s sys 0.000s
I20260812 19:08:16.168048 29281 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:16.173493 2817 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:16.168602 2828 consensus_queue.cc:1046] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 19:08:16.173789 2637 leader_election.cc:288] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.062446 29337 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:16.174280 2613 consensus_queue.cc:1046] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 19:08:16.175351 2637 raft_consensus.cc:3032] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
W20260812 19:08:16.058307 2877 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260812 19:08:16.017840 2358 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:08:16.062006 1986 consensus_meta.cc:287] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.532s user 0.003s sys 0.011s
I20260812 19:08:16.092542 2905 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:16.086937 2805 consensus_meta.cc:287] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.577s user 0.007s sys 0.009s
W20260812 19:08:16.072060 2930 log.cc:924] Time spent T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
I20260812 19:08:16.066256 2908 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.180658 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:16.189007 2913 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:16.189507 2919 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 19:08:16.179643 2742 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:16.076383 2891 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.186084 2905 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:16.190724 2938 log.cc:924] Time spent T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.172s user 0.000s sys 0.001s
I20260812 19:08:16.087510 2904 raft_consensus.cc:482] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:16.082537 2804 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.177685 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:16.192368 29575 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:16.175784 2828 raft_consensus.cc:3032] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Stepping down as leader of term 11
I20260812 19:08:16.068462 2777 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:16.195021 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:16.202206 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:16.200215 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.183813 2358 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:16.176203 2637 raft_consensus.cc:729] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:16.233103 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.233259 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:16.208194 30686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34288 (request call id 934) took 2148 ms (client timeout 1542 ms). Trace:
I20260812 19:08:16.185258 2657 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:16.060259 2892 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.222077 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.222127 29567 raft_consensus.cc:2379] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:16.218984 29580 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:16.223897 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.225140 29577 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:16.261168 2892 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.263228 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.264473 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62647baf8dbe49b9b57e1f296bfa85e6" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.265478 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.194849 2710 consensus_queue.cc:1035] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260812 19:08:16.218494 2904 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:16.270393 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.277909 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.278654 29341 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:16.279340 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.282092 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.283314 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.289163 2891 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.188271 1986 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.194684 2805 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.200112 2647 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:16.234117 30686 rpcz_store.cc:269] 0812 19:08:14.052644 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.094108 (+ 41464us) service_pool.cc:227] Handling call
0812 19:08:16.200500 (+2106392us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003502d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:16.200634 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2657041280}
I20260812 19:08:16.201746 2919 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.243345 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.244841 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:16.247323 2931 log.cc:924] Time spent T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260812 19:08:16.247104 2615 consensus_queue.cc:1046] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260812 19:08:16.221344 2828 raft_consensus.cc:729] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:16.252127 2195 consensus_meta.cc:287] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.508s user 0.005s sys 0.005s
I20260812 19:08:16.210876 2742 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.253021 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:16.191751 2451 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
W20260812 19:08:16.253906 2780 consensus_meta.cc:287] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.526s user 0.010s sys 0.000s
I20260812 19:08:16.237044 2777 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.234386 2704 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:16.240307 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.526010 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.299024 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.526824 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.228425 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:16.284370 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.485091 30676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:16.528258 30676 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:16.511339 30674 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:16.305908 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:16.515178 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 330625us
I20260812 19:08:16.528578 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:16.528913 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 226826us
I20260812 19:08:16.530076 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.338199 2891 raft_consensus.cc:2781] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:16.538813 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.539883 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.540649 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.506254 30671 raft_consensus.cc:3032] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:16.541555 30671 raft_consensus.cc:729] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:16.466271 2908 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:08:16.531060 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 293462us
I20260812 19:08:16.359258 29338 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:16.299463 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.542796 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.546908 29332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.552429 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.553292 29337 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:16.542402 30671 consensus_queue.cc:260] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.510892 2919 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:16.576624 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.576968 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.585457 29331 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:16.587940 30674 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:16.588923 29575 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260812 19:08:16.205619 2913 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.599968 2358 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:16.587462 2947 log.cc:924] Time spent T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
I20260812 19:08:16.321350 2625 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.548254 2610 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:16.602583 2647 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.510170 2132 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.609498 2703 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.566749 2828 consensus_queue.cc:260] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.612664 2777 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:16.524483 2780 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.557507 1986 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.619944 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.623942 29575 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:16.697254 29579 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 913) took 2096 ms. Trace:
I20260812 19:08:16.656354 30676 raft_consensus.cc:2454] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 11.
I20260812 19:08:16.538671 2891 raft_consensus.cc:482] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.699652 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 985) took 1463 ms. Trace:
W20260812 19:08:16.715678 29399 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.012s sys 0.032s
W20260812 19:08:16.718910 29399 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.523s user 0.012s sys 0.032s
I20260812 19:08:16.718340 30684 rpcz_store.cc:276] 0812 19:08:15.236692 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.237187 (+ 495us) service_pool.cc:227] Handling call
0812 19:08:16.699463 (+1462276us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b54005eecd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:16.699594 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1000156928}
W20260812 19:08:16.719228 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 538618us
I20260812 19:08:16.518141 2195 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.614317 2377 raft_consensus.cc:3032] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:16.716466 29579 rpcz_store.cc:276] 0812 19:08:14.601029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.694332 (+ 93303us) service_pool.cc:227] Handling call
0812 19:08:14.796518 (+102186us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:15.372255 (+575737us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.697002 (+1324747us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b540009f454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.697184 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1295459072}
I20260812 19:08:16.668437 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.683868 29580 raft_consensus.cc:1226] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 12 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260812 19:08:16.720532 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:16.716466 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:16.672350 2899 consensus_meta.cc:287] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.585s user 0.010s sys 0.000s
I20260812 19:08:16.740612 30679 raft_consensus.cc:1226] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:16.742058 29567 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:16.744014 30681 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:16.586526 2405 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.144s
W20260812 19:08:16.701843 2369 consensus_meta.cc:287] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.654s user 0.009s sys 0.004s
I20260812 19:08:16.672201 2861 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.716892 2610 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:16.695590 2742 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.577455 2804 consensus_queue.cc:1046] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.161s
I20260812 19:08:16.671590 2952 raft_consensus.cc:482] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.772032 2952 raft_consensus.cc:504] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.774248 2952 leader_election.cc:288] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.729907 2891 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:16.771394 2132 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:16.778393 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57270 (request call id 2351) took 1779 ms. Trace:
I20260812 19:08:16.778811 29576 rpcz_store.cc:276] 0812 19:08:14.998800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.037215 (+ 38415us) service_pool.cc:227] Handling call
0812 19:08:15.041549 (+ 4334us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:15.680391 (+638842us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.778134 (+1097743us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400177854. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.778329 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1398066944}
I20260812 19:08:16.779668 2625 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.771958 2283 raft_consensus.cc:3032] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:16.781916 2283 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:16.584873 2908 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.733778 2377 raft_consensus.cc:729] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:16.782900 2283 consensus_queue.cc:260] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.739234 2828 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Advancing to term 12
I20260812 19:08:16.671249 2792 consensus_queue.cc:1046] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260812 19:08:16.761128 2369 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.771227 29577 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 928) took 1092 ms. Trace:
I20260812 19:08:16.805805 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.805626 29577 rpcz_store.cc:276] 0812 19:08:15.679175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.753622 (+ 74447us) service_pool.cc:227] Handling call
0812 19:08:15.754022 (+ 400us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:16.259323 (+505301us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.770942 (+511619us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540012be54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.771156 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1083038976}
W20260812 19:08:16.759459 2687 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s user 0.028s sys 0.030s
I20260812 19:08:16.734228 2647 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:16.784870 2742 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
W20260812 19:08:16.798364 30670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 976) took 2284 ms (client timeout 1532 ms). Trace:
I20260812 19:08:16.763875 2610 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:16.858969 30676 raft_consensus.cc:1226] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260812 19:08:16.804755 29334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 961) took 1024 ms. Trace:
W20260812 19:08:16.859822 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 45230.3us
I20260812 19:08:16.809198 30686 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:16.860518 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:16.794569 2905 consensus_meta.cc:287] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.562s user 0.003s sys 0.009s
I20260812 19:08:16.810644 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.859730 29334 rpcz_store.cc:276] 0812 19:08:15.779876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.780033 (+ 157us) service_pool.cc:227] Handling call
0812 19:08:15.782080 (+ 2047us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:15.809132 (+ 27052us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:15.809165 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:15.809204 (+ 39us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:15.809234 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.804515 (+995281us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b5400231ed4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.804563 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:16.804692 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":61738368}
I20260812 19:08:16.862319 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.863438 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.795814 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.752144 2899 raft_consensus.cc:504] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.818156 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:16.840160 2283 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:16.842150 2647 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.765653 2861 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.801555 2804 consensus_queue.cc:1225] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Peer dd7e8c7a8bc54e73a39411cfc241fe9c log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260812 19:08:16.813303 30670 rpcz_store.cc:269] 0812 19:08:14.513774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:14.556036 (+ 42262us) service_pool.cc:227] Handling call
0812 19:08:16.798128 (+2242092us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400530f54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:16.798286 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2797733632}
I20260812 19:08:16.849068 2601 consensus_queue.cc:1046] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:08:16.799254 2956 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 19:08:16.892484 2956 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:16.893174 2956 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:16.894472 2899 leader_election.cc:288] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.838564 2778 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:16.854723 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 929) took 1024 ms. Trace:
I20260812 19:08:16.854748 2703 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:16.797464 2377 consensus_queue.cc:260] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.899940 2377 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:16.900738 2647 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.861132 29339 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 7.
I20260812 19:08:16.902735 2601 consensus_queue.cc:1225] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer dd7e8c7a8bc54e73a39411cfc241fe9c log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260812 19:08:16.903363 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.903642 2601 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:16.904289 29339 raft_consensus.cc:2354] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:16.904525 2601 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.905426 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.906355 2861 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.899500 29572 rpcz_store.cc:276] 0812 19:08:15.830687 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.906529 (+ 75842us) service_pool.cc:227] Handling call
0812 19:08:15.906852 (+ 323us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:15.907193 (+ 341us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:15.907221 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:15.907676 (+ 455us) log.cc:842] Serialized 26 byte log entry
0812 19:08:16.854124 (+946448us) spinlock_profiling.cc:243] Waited 7.24 ms on lock 0x7b1800461b60. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c69a9d82c 00007f9c66ed49dc 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb 00007f9c69a95063 00007f9c69a95411
0812 19:08:16.854279 (+ 155us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:16.854313 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.854396 (+ 83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:16.854443 (+ 47us) raft_consensus.cc:1671] finished
0812 19:08:16.854453 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:16.854662 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1472,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":43260,"spinlock_wait_cycles":16642304,"thread_start_us":71378,"threads_started":1,"wal-append.queue_time_us":304342}
W20260812 19:08:16.888250 2904 consensus_meta.cc:287] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.565s user 0.011s sys 0.000s
I20260812 19:08:16.910502 2904 raft_consensus.cc:504] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.911702 2780 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.906203 2804 raft_consensus.cc:3032] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:16.912552 2804 raft_consensus.cc:729] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:16.864225 29530 leader_election.cc:334] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.510s (SENT)
I20260812 19:08:16.864727 29281 leader_election.cc:302] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:08:16.810895 2687 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s user 0.028s sys 0.031s
I20260812 19:08:16.866041 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.848977 2195 leader_election.cc:288] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.871711 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.913718 2804 consensus_queue.cc:260] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.872164 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:16.872349 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.915539 30686 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:16.915683 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 997) took 1310 ms. Trace:
I20260812 19:08:16.876204 2908 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.871964 2369 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.866657 2905 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.864530 2742 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:16.880793 2913 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.919337 2905 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.882884 30629 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:16.922191 2924 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:16.923077 2924 raft_consensus.cc:686] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [term 5 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:16.916253 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c89f8a2f65164cd08d1e760f9005e496" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.914937 29530 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:16.915963 2804 raft_consensus.cc:3037] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:16.891183 29342 raft_consensus.cc:2379] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 7: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:16.928691 2601 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.929358 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.929502 30686 raft_consensus.cc:2454] T 637fa5c7fa824870a0da61e2bc8400af P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:16.931927 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 989) took 1631 ms. Trace:
I20260812 19:08:16.919915 2742 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:16.936748 30675 rpcz_store.cc:276] 0812 19:08:15.604961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.901585 (+296624us) service_pool.cc:227] Handling call
0812 19:08:15.901852 (+ 267us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:15.987932 (+ 86080us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.915474 (+927542us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b540070e9d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.915630 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3932032}
I20260812 19:08:16.939218 2899 raft_consensus.cc:2781] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader election won for term 11
W20260812 19:08:16.938860 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 74212.1us
I20260812 19:08:16.924750 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.940758 29281 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:16.938656 30685 rpcz_store.cc:276] 0812 19:08:15.300061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.343323 (+ 43262us) service_pool.cc:227] Handling call
0812 19:08:15.343629 (+ 306us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:15.989819 (+646190us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:16.931677 (+941858us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540067d9d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:16.931869 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1285650688}
W20260812 19:08:16.941778 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 77588.9us
W20260812 19:08:16.944317 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 443790us
I20260812 19:08:16.947443 30685 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260812 19:08:16.947798 30685 pending_rounds.cc:76] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Aborting all ops after (but not including) 7
I20260812 19:08:16.947996 30685 pending_rounds.cc:97] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260812 19:08:16.924726 2924 consensus_queue.cc:237] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.948271 30685 raft_consensus.cc:2866] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:16.952214 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:16.953059 29576 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:16.957000 29572 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260812 19:08:16.959906 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:16.940732 2904 leader_election.cc:288] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.979300 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:16.942049 2899 raft_consensus.cc:686] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:16.985071 29281 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:16.950316 2785 raft_consensus.cc:2726] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:16.987258 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:16.988724 29530 leader_election.cc:334] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.535s (SENT)
I20260812 19:08:16.989183 29530 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:16.991741 2358 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:16.992322 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.993583 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:16.978094 2195 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:17.001489 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 260512us
I20260812 19:08:17.011687 29399 leader_election.cc:302] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.016067 29399 leader_election.cc:302] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.017692 29326 raft_consensus.cc:2396] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:08:17.018100 29399 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:17.018555 29530 leader_election.cc:334] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.528s (SENT)
I20260812 19:08:17.018985 29530 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.019585 29399 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.019992 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.007516 2377 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:16.997936 2489 raft_consensus.cc:2726] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:17.027217 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:17.030202 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 227428us
I20260812 19:08:16.977737 2660 consensus_queue.cc:1046] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260812 19:08:17.002537 2817 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:17.018240 2904 raft_consensus.cc:2726] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:17.041601 2489 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:08:17.030277 2785 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:17.062655 29399 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:17.033419 2742 raft_consensus.cc:2726] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:17.034324 2377 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.001916 2848 consensus_queue.cc:1046] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260812 19:08:17.050068 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.045621 2899 consensus_queue.cc:237] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.051028 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 998) took 1445 ms. Trace:
I20260812 19:08:17.051893 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1039) took 1027 ms. Trace:
I20260812 19:08:17.053241 2919 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:16.998396 2879 consensus_queue.cc:1046] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260812 19:08:17.054854 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.056263 2891 raft_consensus.cc:504] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.034737 2913 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:17.030258 2905 raft_consensus.cc:2726] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:17.046550 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.107836 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.108530 29341 raft_consensus.cc:2454] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:17.109364 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.012815 2623 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260812 19:08:17.048270 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.071606 2874 raft_consensus.cc:3032] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:17.040058 2908 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
I20260812 19:08:17.031355 2968 raft_consensus.cc:482] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:17.077085 2769 consensus_meta.cc:287] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.033s user 0.010s sys 0.002s
I20260812 19:08:17.070716 2904 raft_consensus.cc:2781] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:17.078830 29399 leader_election.cc:302] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:17.079360 30674 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:17.080232 29332 raft_consensus.cc:2396] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260812 19:08:17.091812 30681 rpcz_store.cc:276] 0812 19:08:16.024007 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.549152 (+525145us) service_pool.cc:227] Handling call
0812 19:08:16.555543 (+ 6391us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:16.750653 (+195110us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.051680 (+301027us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400731ed4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:17.051839 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":424792320}
I20260812 19:08:17.091607 30679 rpcz_store.cc:276] 0812 19:08:15.605226 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.905753 (+300527us) service_pool.cc:227] Handling call
0812 19:08:15.906077 (+ 324us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:16.750479 (+844402us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.050792 (+300313us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400764dd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:17.050970 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1918868224}
I20260812 19:08:17.094007 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:17.096427 29403 leader_election.cc:334] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.507s (SENT)
I20260812 19:08:17.048805 2817 raft_consensus.cc:686] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.111408 30629 leader_election.cc:302] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:17.142453 30686 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260812 19:08:17.094928 2645 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s user 0.020s sys 0.012s
W20260812 19:08:17.113360 29530 leader_election.cc:334] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.588s (SENT)
I20260812 19:08:17.113675 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.147912 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.070217 2956 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.082039 2610 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.115401 29459 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:17.116299 29452 raft_consensus.cc:1226] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260812 19:08:17.113376 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.154932 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.155539 29446 raft_consensus.cc:3037] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:17.102342 2688 consensus_queue.cc:1046] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260812 19:08:17.114431 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.133013 29450 raft_consensus.cc:1226] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:17.132995 29458 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
W20260812 19:08:17.140149 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 43787us
I20260812 19:08:17.160799 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.161583 29339 raft_consensus.cc:2379] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.161477 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:17.161742 29403 leader_election.cc:334] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.590s (SENT)
I20260812 19:08:17.162310 29335 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
W20260812 19:08:17.162797 29403 leader_election.cc:334] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.605s (SENT)
I20260812 19:08:17.163105 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:17.146875 2645 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s user 0.020s sys 0.012s
I20260812 19:08:17.137326 2904 raft_consensus.cc:482] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:17.183063 29450 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:17.149979 29530 leader_election.cc:302] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.183455 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d3af04408974eeab695361c3b2a8410" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.184374 29530 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.185621 29530 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:17.118263 29445 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
W20260812 19:08:17.158197 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 46907.1us
I20260812 19:08:17.159365 29452 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260812 19:08:17.150184 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.163230 29403 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:17.148109 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 69356.5us
W20260812 19:08:17.182226 29342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2431) took 1159 ms (client timeout 1535 ms). Trace:
I20260812 19:08:17.117774 2874 raft_consensus.cc:729] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.192157 29452 raft_consensus.cc:1226] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260812 19:08:17.152979 2913 raft_consensus.cc:2781] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:17.186122 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.199662 29335 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 9.
W20260812 19:08:17.190490 29403 leader_election.cc:334] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.669s (SENT)
I20260812 19:08:17.200696 29458 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260812 19:08:17.200920 29403 leader_election.cc:334] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295): Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.739s (SENT)
I20260812 19:08:17.201269 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.134030 2792 consensus_queue.cc:1046] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260812 19:08:17.190649 29342 rpcz_store.cc:269] 0812 19:08:16.004660 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.004776 (+ 116us) service_pool.cc:227] Handling call
0812 19:08:17.164017 (+1159241us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b5400218254. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:17.164138 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1909222784}
I20260812 19:08:17.202606 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.188432 29453 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:17.203753 2828 raft_consensus.cc:3032] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:17.204196 2828 raft_consensus.cc:729] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.203406 2874 consensus_queue.cc:260] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:17.205597 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.185925 29446 raft_consensus.cc:2454] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:17.201376 29403 leader_election.cc:302] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.205464 29459 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:17.207563 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.165339 2791 raft_consensus.cc:2781] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:17.214435 29457 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 11: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.204818 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.209518 29452 raft_consensus.cc:1226] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260812 19:08:17.209225 29403 leader_election.cc:302] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.216281 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.200623 2377 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.142092 2769 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.153182 2879 consensus_queue.cc:1046] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260812 19:08:17.224617 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.227377 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.226056 2919 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.222468 2623 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:17.229928 29452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.272156 2974 raft_consensus.cc:2726] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260812 19:08:17.275689 2988 log.cc:924] Time spent T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.123s user 0.000s sys 0.001s
I20260812 19:08:17.269889 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.370528 29453 raft_consensus.cc:3037] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
W20260812 19:08:17.211992 2978 log.cc:924] Time spent T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260812 19:08:17.277375 2687 raft_consensus.cc:3032] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:17.266952 2828 consensus_queue.cc:260] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.279050 2956 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 11 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.138985 2968 raft_consensus.cc:504] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.226346 2660 raft_consensus.cc:3032] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:17.160020 2977 consensus_queue.cc:1035] T b83150d7d32c400083ab43de4f67f607 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260812 19:08:17.286191 30670 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:17.267618 2817 consensus_queue.cc:237] T 637fa5c7fa824870a0da61e2bc8400af P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.199278 2913 raft_consensus.cc:482] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:17.268458 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.228644 29403 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.189489 2904 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:17.222078 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:17.362264 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 58166.5us
I20260812 19:08:17.198135 2905 raft_consensus.cc:3037] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.192703 2975 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:17.242362 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:17.240411 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 54856.6us
I20260812 19:08:17.431339 29530 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.226570 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.290839 2891 leader_election.cc:288] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.227739 2791 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:17.367591 2623 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.276086 1362 raft_consensus.cc:3032] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:17.362040 30671 raft_consensus.cc:3037] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:17.408252 2660 raft_consensus.cc:729] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.380523 2615 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:17.382650 29453 raft_consensus.cc:2454] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9.
I20260812 19:08:17.383414 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.387005 29339 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
I20260812 19:08:17.452979 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.188251 2972 raft_consensus.cc:2726] T 62647baf8dbe49b9b57e1f296bfa85e6 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:17.455190 2972 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:17.455328 2752 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:17.274911 2780 raft_consensus.cc:2726] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:17.293298 2985 log.cc:924] Time spent T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.071s user 0.000s sys 0.002s
I20260812 19:08:17.300222 2804 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:17.269038 2874 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:17.456818 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7f7d48e270c456b9b7eeab6f3165a89" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.409653 29282 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.201766 2778 consensus_queue.cc:1046] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.189s
I20260812 19:08:17.320230 2769 leader_election.cc:288] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:17.457659 29335 raft_consensus.cc:2396] T c7f7d48e270c456b9b7eeab6f3165a89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260812 19:08:17.414999 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 186410us
I20260812 19:08:17.388885 2687 raft_consensus.cc:729] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.466585 2687 consensus_queue.cc:260] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.467765 2687 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:17.455721 2972 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.470685 2876 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:17.472776 2972 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.442440 29458 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.473224 2623 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:17.376289 2645 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:08:17.478024 2637 consensus_queue.cc:260] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:17.480295 2637 raft_consensus.cc:3037] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:17.449456 29446 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 10: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.448374 1362 raft_consensus.cc:729] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.482897 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.482815 1362 consensus_queue.cc:260] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:17.483990 1362 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:17.484377 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.412225 2377 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:17.486208 2993 log.cc:924] Time spent T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.073s user 0.000s sys 0.002s
I20260812 19:08:17.486435 2377 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.487919 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.489382 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.221233 2967 consensus_queue.cc:1046] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.249s
I20260812 19:08:17.491072 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.491007 2615 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.451136 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.411726 2465 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:17.497473 2465 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.499483 30671 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260812 19:08:17.499840 30671 pending_rounds.cc:76] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Aborting all ops after (but not including) 8
I20260812 19:08:17.500068 30671 pending_rounds.cc:97] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260812 19:08:17.500303 30671 raft_consensus.cc:2866] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:17.453313 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.500916 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1032) took 2291 ms. Trace:
I20260812 19:08:17.501242 29339 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:17.502233 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.502784 29339 raft_consensus.cc:2454] T b83150d7d32c400083ab43de4f67f607 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 11.
I20260812 19:08:17.503512 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.504973 30629 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.439715 2782 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:17.432704 2956 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:17.445807 2791 raft_consensus.cc:3037] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.454450 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.455538 2968 leader_election.cc:288] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.297008 2610 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.427579 2913 raft_consensus.cc:3037] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:17.457269 2804 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.465004 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2472) took 1180 ms. Trace:
I20260812 19:08:17.485533 2379 raft_consensus.cc:3032] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:17.513202 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.495754 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:17.507910 3003 log.cc:924] Time spent T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.054s user 0.003s sys 0.001s
I20260812 19:08:17.518369 29446 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:17.418264 2879 consensus_queue.cc:1046] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260812 19:08:17.465839 2778 raft_consensus.cc:3032] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:17.507261 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.506990 2777 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:17.522995 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.523854 29446 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.513392 29403 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:17.515722 29335 rpcz_store.cc:276] 0812 19:08:16.284613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.456578 (+1171965us) service_pool.cc:227] Handling call
0812 19:08:17.464928 (+ 8350us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.528640 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.529816 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:17.495442 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 85887.2us
I20260812 19:08:17.517681 29458 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:17.530947 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.534312 2876 consensus_queue.cc:1046] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260812 19:08:17.510725 29453 raft_consensus.cc:2379] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 5: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.536304 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.520788 29459 raft_consensus.cc:2379] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.541985 29341 raft_consensus.cc:2454] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:08:17.501214 30671 rpcz_store.cc:276] 0812 19:08:15.209976 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.287594 (+ 77618us) service_pool.cc:227] Handling call
0812 19:08:15.287926 (+ 332us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:16.506151 (+1218225us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b54005c2720. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d9e730 00007f9c69e10300 00007f9c69e11371 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc
0812 19:08:16.769617 (+263466us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b58007925b0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d89f63 00007f9c69dedaba 00007f9c69e053af 00007f9c69e06b4e 00007f9c69e10c89 00007f9c69e11556 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684
0812 19:08:17.308165 (+538548us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54005c2720. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69d9c40d 00007f9c69d8a050 00007f9c69dedaba 00007f9c69e053af 00007f9c69e06b4e 00007f9c69e10c89 00007f9c69e11556 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9
0812 19:08:17.500698 (+192533us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.500856 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":314906496}
I20260812 19:08:17.545382 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.541298 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.547488 29446 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.548272 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7adcd8cac12a4350a441ca5a35090ec9" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.548960 29446 raft_consensus.cc:2379] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:17.516993 2379 raft_consensus.cc:729] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.552920 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.554623 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.517494 2377 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.567051 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.561427 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.562201 29282 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.568666 30685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.562318 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.569267 30685 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:17.569800 29335 raft_consensus.cc:2379] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.569205 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.573065 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.573843 30674 raft_consensus.cc:2379] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 4: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.578768 30625 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.580821 30625 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:17.582135 30625 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:17.522210 2778 raft_consensus.cc:729] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:17.584019 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.456748 2660 consensus_queue.cc:260] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.585105 30684 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.534587 2617 consensus_queue.cc:1046] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:08:17.583952 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.594681 29341 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 11: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:17.595057 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2351) took 1060 ms. Trace:
I20260812 19:08:17.595378 29341 rpcz_store.cc:276] 0812 19:08:16.534900 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.583675 (+1048775us) service_pool.cc:227] Handling call
0812 19:08:17.595000 (+ 11325us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.540378 2777 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:17.542794 2968 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:17.511348 2782 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.557267 2465 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.585932 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.588754 30625 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:17.690850 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 102176us
I20260812 19:08:17.609750 2777 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.594002 29458 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:17.596202 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.697329 29577 raft_consensus.cc:1226] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:17.697780 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:17.586917 2706 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:17.557222 2878 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:17.607602 29403 leader_election.cc:302] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:17.706398 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.707132 29458 raft_consensus.cc:2379] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.592654 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
W20260812 19:08:17.590588 29569 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46208 (request call id 923) took 2027 ms (client timeout 1754 ms). Trace:
I20260812 19:08:17.591146 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.704306 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.708117 30674 raft_consensus.cc:2379] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.586808 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.618048 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.560325 2792 consensus_queue.cc:1046] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260812 19:08:17.583666 2623 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:17.643774 29579 raft_consensus.cc:1261] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260812 19:08:17.712291 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.688108 2660 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.669785 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.713930 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ea822fc3c0f456ab7c3f8d71f84ff27" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:17.665760 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 235806us
I20260812 19:08:17.646806 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69508446a398466a943e7ad290aa8e0c" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.649757 29575 raft_consensus.cc:1226] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 LEADER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:17.629119 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.648355 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0dfbce6d2e3431ebc71db36ae455105" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.716794 2465 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260812 19:08:17.721683 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.722379 29457 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:17.725272 29332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33256 (request call id 2465) took 1442 ms. Trace:
I20260812 19:08:17.712633 2878 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.728250 29579 raft_consensus.cc:1226] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:17.728505 29156 maintenance_manager.cc:419] P f47470148eaf49acb3f5b4813eb35f3e: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033398
I20260812 19:08:17.729038 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.698298 2283 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:17.729765 29450 raft_consensus.cc:2354] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:17.729915 2283 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.730818 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.732070 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.732585 29450 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:17.738951 2611 raft_consensus.cc:3032] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Stepping down as leader of term 4
I20260812 19:08:17.695145 2594 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:17.639945 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:17.682427 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 251171us
I20260812 19:08:17.684630 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.650652 29576 raft_consensus.cc:1226] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260812 19:08:17.593720 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d05c973d4068462fa79a7bec57fc6e89" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.590793 2377 raft_consensus.cc:2781] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:17.589630 2861 raft_consensus.cc:2726] T c89f8a2f65164cd08d1e760f9005e496 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:17.765661 30676 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260812 19:08:17.617058 2707 raft_consensus.cc:2781] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:17.616742 2968 raft_consensus.cc:482] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:17.703569 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.705402 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.770524 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.771215 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.772156 29335 raft_consensus.cc:2396] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260812 19:08:17.708900 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.708916 30679 raft_consensus.cc:2379] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
W20260812 19:08:17.707700 29569 rpcz_store.cc:269] 0812 19:08:15.562989 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:15.563432 (+ 443us) service_pool.cc:227] Handling call
0812 19:08:17.590423 (+2026991us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400157054. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:17.590522 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":436843776}
I20260812 19:08:17.777479 29445 raft_consensus.cc:2454] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 6.
I20260812 19:08:17.777880 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.712062 2974 raft_consensus.cc:2781] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 19:08:17.654675 2782 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.710439 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.786365 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.710923 29459 raft_consensus.cc:2379] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 10: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.709002 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.660164 2804 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:17.785476 29578 raft_consensus.cc:2454] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:08:17.790337 29575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.589382 2688 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260812 19:08:17.739279 29574 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.725731 29332 rpcz_store.cc:276] 0812 19:08:16.282488 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.546675 (+264187us) service_pool.cc:227] Handling call
0812 19:08:17.725055 (+1178380us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54003773d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:17.725204 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1215039232}
I20260812 19:08:17.741745 29334 raft_consensus.cc:2396] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260812 19:08:17.817735 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.645538 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.669354 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.818464 30686 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:17.716825 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.701329 29399 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:17.819517 29451 raft_consensus.cc:2354] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
W20260812 19:08:17.819592 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 118340us
I20260812 19:08:17.820326 29399 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:17.767470 29530 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.822467 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.822177 30625 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:17.822815 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 55457.6us
I20260812 19:08:17.823598 29572 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:17.824188 30625 leader_election.cc:302] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:17.746415 29331 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 11: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:17.792622 2974 raft_consensus.cc:482] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:17.768286 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.770721 29341 raft_consensus.cc:2396] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260812 19:08:17.656792 2615 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:17.706995 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 99473.6us
I20260812 19:08:17.779968 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:17.782120 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.782574 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.886866 29577 raft_consensus.cc:2379] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.763396 29457 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 5.
I20260812 19:08:17.821074 2876 raft_consensus.cc:482] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:17.708743 2891 consensus_queue.cc:1035] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
I20260812 19:08:17.895040 29457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.781194 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.896754 30679 raft_consensus.cc:2379] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 9: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.681114 2379 consensus_queue.cc:260] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.796914 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.814746 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd63b9965cdb4b568fd3b2860f8c2e61" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.900228 29576 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 6: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:17.900360 29445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.816025 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.816319 29578 raft_consensus.cc:1261] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260812 19:08:17.779227 2968 raft_consensus.cc:3037] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:17.748405 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:17.827499 29332 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:17.828644 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.830264 30625 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:17.908417 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 78231.1us
I20260812 19:08:17.787050 29569 raft_consensus.cc:2454] T d28ed56ce03a49908ad6f468e43b5402 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 5.
I20260812 19:08:17.830442 29580 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:17.585732 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.790284 2377 raft_consensus.cc:686] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.920210 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e4ba354bddd43ec97a890c3bb577330" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.791769 2707 raft_consensus.cc:686] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [term 9 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:17.830080 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.703945 2778 consensus_queue.cc:260] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.862504 29528 leader_election.cc:398] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 election: Vote denied by peer 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659) with higher term. Message: Invalid argument: T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
W20260812 19:08:17.923693 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 61324.1us
I20260812 19:08:17.924206 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.834347 29399 leader_election.cc:302] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:17.925352 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 93219.1us
I20260812 19:08:17.926298 29399 leader_election.cc:302] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.864660 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2344) took 1579 ms. Trace:
I20260812 19:08:17.865486 30686 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 11.
I20260812 19:08:17.870151 2878 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:17.861712 2804 raft_consensus.cc:2781] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:17.833243 2789 raft_consensus.cc:2726] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:17.936990 29527 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:17.872999 3009 log.cc:924] Time spent T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260812 19:08:17.830458 2596 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:17.830052 2956 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election won for term 6
I20260812 19:08:17.882964 29338 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:17.883930 29339 raft_consensus.cc:2454] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:17.855432 30684 raft_consensus.cc:2379] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 5: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:17.942502 29332 raft_consensus.cc:595] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:17.942907 29332 raft_consensus.cc:603] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:17.833078 29403 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.881677 2791 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:17.783483 2283 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:17.891938 29335 raft_consensus.cc:2379] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 11: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:17.893358 29577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47762c70d86d4362baf4ba75331efe8a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.739673 2611 raft_consensus.cc:729] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:17.897460 29457 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:17.788117 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1226d05dc64043078e42cf470348655d" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.901165 29445 raft_consensus.cc:2379] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.829609 2959 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 19:08:17.901502 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.910064 30625 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:17.953537 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 43587.8us
I20260812 19:08:17.953756 30679 raft_consensus.cc:1226] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 4. Current term is 5. Ops: []
I20260812 19:08:17.825042 29530 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.911322 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.912782 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39e2c7b7710843cf96e72ac710028fa9" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.833884 2974 raft_consensus.cc:3037] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Advancing to term 12
I20260812 19:08:17.921469 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2343) took 1636 ms. Trace:
I20260812 19:08:17.924495 29528 leader_election.cc:302] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:17.927078 29341 rpcz_store.cc:276] 0812 19:08:16.285704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.863713 (+1578009us) service_pool.cc:227] Handling call
0812 19:08:17.864065 (+ 352us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.864260 (+ 195us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.864284 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:17.864326 (+ 42us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:17.864371 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.864465 (+ 94us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:17.864588 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.927744 30686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "647dfc8bad934787a924a06988dda474" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.929323 29580 raft_consensus.cc:2454] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:17.917807 29450 raft_consensus.cc:2454] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 4.
I20260812 19:08:17.922063 29574 raft_consensus.cc:2454] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:17.959589 29335 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 10. Current term is 11. Ops: []
I20260812 19:08:17.930347 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.960305 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2362) took 1422 ms. Trace:
I20260812 19:08:17.930917 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:17.960598 29450 raft_consensus.cc:1261] T 8cbee78e8845445eb3a4cbba1399e988 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260812 19:08:17.961058 29459 raft_consensus.cc:2379] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 8: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:17.931174 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe3191acdfe44a019be79eda30b9e79b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:17.961796 30681 raft_consensus.cc:2396] T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 for term 10 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260812 19:08:17.933979 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:17.945219 29458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:17.947734 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2345) took 1620 ms. Trace:
I20260812 19:08:17.934618 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebf3eea25534182aa1c9f148898dc8b" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:17.923237 2594 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260812 19:08:17.955668 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 130681us
I20260812 19:08:17.962507 29338 rpcz_store.cc:276] 0812 19:08:16.327712 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.863229 (+1535517us) service_pool.cc:227] Handling call
0812 19:08:17.882751 (+ 19522us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.947459 (+ 64708us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.947668 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.963233 29335 rpcz_store.cc:276] 0812 19:08:16.539324 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.958374 (+1419050us) service_pool.cc:227] Handling call
0812 19:08:17.958665 (+ 291us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.960037 (+ 1372us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.960246 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1622400}
I20260812 19:08:17.965772 29530 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.957901 29331 rpcz_store.cc:276] 0812 19:08:16.285442 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.834244 (+1548802us) service_pool.cc:227] Handling call
0812 19:08:17.920965 (+ 86721us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.921160 (+ 195us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.921185 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:17.921225 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:17.921270 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.921318 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:17.921411 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.956777 29457 raft_consensus.cc:1226] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:17.930404 2919 raft_consensus.cc:2726] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:08:17.960412 29578 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:17.978917 29445 raft_consensus.cc:1226] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:17.975415 30679 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
W20260812 19:08:17.949160 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 243344us
I20260812 19:08:17.976514 29341 raft_consensus.cc:1226] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
W20260812 19:08:17.977504 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 53094.5us
I20260812 19:08:17.980589 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33260 (request call id 2363) took 1397 ms. Trace:
I20260812 19:08:17.980933 29341 rpcz_store.cc:276] 0812 19:08:16.582822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.958945 (+1376123us) service_pool.cc:227] Handling call
0812 19:08:17.959235 (+ 290us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.980333 (+ 21098us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.980530 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.984076 29445 raft_consensus.cc:1226] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:17.984984 29530 leader_election.cc:398] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:17.985324 29530 leader_election.cc:401] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260812 19:08:17.986426 29530 leader_election.cc:302] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:17.990096 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 970) took 1274 ms. Trace:
I20260812 19:08:17.990522 29341 rpcz_store.cc:276] 0812 19:08:16.716087 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.989228 (+1273141us) service_pool.cc:227] Handling call
0812 19:08:17.989553 (+ 325us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.989735 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.989761 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:17.989804 (+ 43us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:17.989850 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.989901 (+ 51us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:17.990034 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.991907 29341 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:17.992494 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 973) took 1069 ms. Trace:
I20260812 19:08:17.992777 29341 rpcz_store.cc:276] 0812 19:08:16.922959 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.991428 (+1068469us) service_pool.cc:227] Handling call
0812 19:08:17.991741 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.992279 (+ 538us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.992444 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.995193 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 968) took 1317 ms. Trace:
I20260812 19:08:17.996047 29335 rpcz_store.cc:276] 0812 19:08:16.679184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.994287 (+1315103us) service_pool.cc:227] Handling call
0812 19:08:17.994676 (+ 389us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.994867 (+ 191us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.994891 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:17.994930 (+ 39us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:17.994977 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.995036 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:17.995132 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:17.902120 2876 raft_consensus.cc:504] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:17.999650 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 975) took 1076 ms. Trace:
I20260812 19:08:17.999774 29445 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:17.999922 29530 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:17.942526 2379 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:17.950133 2785 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:17.941210 2804 raft_consensus.cc:686] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:18.004637 29450 raft_consensus.cc:1261] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:18.005146 29530 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:18.000489 29341 rpcz_store.cc:276] 0812 19:08:16.923642 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.993442 (+1069800us) service_pool.cc:227] Handling call
0812 19:08:17.993717 (+ 275us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.993911 (+ 194us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.993936 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:17.999402 (+ 5466us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:17.999456 (+ 54us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:17.999510 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:17.999601 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:18.011200 30674 raft_consensus.cc:1226] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: []
I20260812 19:08:18.015074 29445 raft_consensus.cc:1261] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260812 19:08:18.015805 2625 consensus_queue.cc:1046] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260812 19:08:18.016664 2862 raft_consensus.cc:3032] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Stepping down as leader of term 3
I20260812 19:08:18.017683 29458 pending_rounds.cc:76] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 6
I20260812 19:08:17.977715 2596 raft_consensus.cc:3037] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:18.038156 3022 consensus_queue.cc:1046] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 19:08:18.038995 2283 consensus_queue.cc:1046] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 19:08:18.039266 2195 raft_consensus.cc:3032] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:18.039707 2195 raft_consensus.cc:729] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:18.039978 2283 raft_consensus.cc:3032] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:18.005380 2786 raft_consensus.cc:2726] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:18.040436 2195 consensus_queue.cc:260] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:18.041534 1362 consensus_meta.cc:287] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.557s user 0.009s sys 0.009s
I20260812 19:08:18.011806 2782 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260812 19:08:18.022585 2687 consensus_meta.cc:287] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.554s user 0.007s sys 0.005s
I20260812 19:08:18.012127 2919 raft_consensus.cc:2726] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:18.027621 30685 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:17.957435 2789 raft_consensus.cc:2726] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:18.033342 29445 raft_consensus.cc:1261] T da12498df45141cd80e5cb36e68775c5 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:18.034541 29331 raft_consensus.cc:1261] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
W20260812 19:08:18.026676 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 44591.4us
I20260812 19:08:18.035575 29458 pending_rounds.cc:97] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260812 19:08:18.048802 29331 pending_rounds.cc:76] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Aborting all ops after (but not including) 9
I20260812 19:08:18.049213 29331 pending_rounds.cc:97] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260812 19:08:18.049479 29331 raft_consensus.cc:2866] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:18.036561 2625 raft_consensus.cc:3032] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:18.037091 2377 consensus_queue.cc:237] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.045431 29338 raft_consensus.cc:1261] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:18.037652 2805 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.041841 2195 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:18.046717 29530 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:18.047344 30684 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:18.000859 2972 raft_consensus.cc:2781] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:18.048293 2786 raft_consensus.cc:2726] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260812 19:08:18.029150 2891 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:18.088181 2972 raft_consensus.cc:482] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:18.088665 2972 raft_consensus.cc:3037] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:18.043062 2791 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:18.062312 2625 raft_consensus.cc:729] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
W20260812 19:08:18.093374 3016 log.cc:924] Time spent T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.193s user 0.000s sys 0.002s
I20260812 19:08:18.092938 2804 consensus_queue.cc:237] T 47762c70d86d4362baf4ba75331efe8a P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.082700 29572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:18.094295 2904 consensus_meta.cc:287] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.648s user 0.003s sys 0.011s
I20260812 19:08:18.094705 2904 raft_consensus.cc:504] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:18.095556 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 141192us
I20260812 19:08:18.096679 2904 leader_election.cc:288] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.092811 2625 consensus_queue.cc:260] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.097308 29456 raft_consensus.cc:1226] T bf300c57bddf44ffb79e424049fe4c52 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
W20260812 19:08:18.079150 3019 log.cc:924] Time spent T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.156s user 0.000s sys 0.001s
I20260812 19:08:18.098604 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2391) took 1040 ms. Trace:
I20260812 19:08:18.099033 29339 rpcz_store.cc:276] 0812 19:08:17.058612 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.503302 (+444690us) service_pool.cc:227] Handling call
0812 19:08:18.098389 (+595087us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54003d8e54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:18.098530 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":873547648}
I20260812 19:08:18.100127 29339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:18.102464 2625 raft_consensus.cc:3037] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:18.104543 29332 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:18.106362 29335 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260812 19:08:18.083220 1986 raft_consensus.cc:3032] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:18.107211 1986 raft_consensus.cc:729] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:18.108029 1986 consensus_queue.cc:260] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.109141 1986 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:18.110761 2610 raft_consensus.cc:3032] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:18.111202 2610 raft_consensus.cc:729] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:18.112108 2610 consensus_queue.cc:260] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.112706 2785 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260812 19:08:18.113234 2610 raft_consensus.cc:3037] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:18.106601 29445 raft_consensus.cc:1226] T 47762c70d86d4362baf4ba75331efe8a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:18.026425 2926 raft_consensus.cc:482] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:18.035436 2862 raft_consensus.cc:729] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 LEADER]: Becoming Follower/Learner. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:18.115406 2862 consensus_queue.cc:260] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.116472 2862 raft_consensus.cc:3037] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Advancing to term 4
I20260812 19:08:18.117269 2687 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260812 19:08:18.085855 2913 consensus_meta.cc:287] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.574s user 0.012s sys 0.000s
I20260812 19:08:18.117837 2687 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.118522 2879 raft_consensus.cc:3032] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:18.119016 29445 raft_consensus.cc:1261] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260812 19:08:18.119798 2687 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.120335 29445 raft_consensus.cc:1261] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260812 19:08:18.121639 29445 raft_consensus.cc:1261] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260812 19:08:18.064378 29450 raft_consensus.cc:1261] T 2743b6275ccc4fbdbe5f71c667adee8b P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260812 19:08:18.064545 2706 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:18.117309 2660 consensus_queue.cc:1035] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260812 19:08:18.124866 2706 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:18.114532 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 64830.4us
I20260812 19:08:18.048465 2707 consensus_queue.cc:237] T ffa1accbf0a749988850cc0f47a8efb0 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.128038 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:18.003805 2645 raft_consensus.cc:2726] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:18.129163 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:18.003350 2465 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 8. Current term is 9.
I20260812 19:08:18.129699 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:18.130571 29458 raft_consensus.cc:2866] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:18.050129 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 969) took 1342 ms. Trace:
W20260812 19:08:18.130236 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 83604.3us
I20260812 19:08:18.142249 29331 rpcz_store.cc:276] 0812 19:08:16.708006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.989047 (+1281041us) service_pool.cc:227] Handling call
0812 19:08:18.034199 (+ 45152us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.049864 (+ 15665us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:18.050077 (+ 213us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:18.143380 29530 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:18.144053 29339 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "69bfa848552945a18495c13ca6707d65"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:18.144624 29530 leader_election.cc:302] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:18.105898 29332 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:18.149868 29281 leader_election.cc:302] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:18.152269 2975 raft_consensus.cc:2726] T c7f7d48e270c456b9b7eeab6f3165a89 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:18.040370 2283 raft_consensus.cc:729] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:18.141798 29335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:18.155329 29530 leader_election.cc:302] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
W20260812 19:08:18.156663 29334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2360) took 1619 ms (client timeout 1889 ms). Trace:
W20260812 19:08:18.137780 2905 consensus_meta.cc:287] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.682s user 0.006s sys 0.006s
I20260812 19:08:18.158181 29530 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:18.157068 29334 rpcz_store.cc:269] 0812 19:08:16.537695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.862519 (+324824us) service_pool.cc:227] Handling call
0812 19:08:18.156461 (+1293942us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b5400012cd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:18.156604 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1901028608}
I20260812 19:08:18.138829 29445 raft_consensus.cc:1261] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260812 19:08:18.118978 2879 raft_consensus.cc:729] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:18.163164 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 236902us
I20260812 19:08:18.163471 29281 leader_election.cc:302] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
W20260812 19:08:18.207849 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 44440.7us
I20260812 19:08:18.210291 29528 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:18.210928 30625 leader_election.cc:302] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:18.211640 29528 leader_election.cc:302] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
W20260812 19:08:18.212598 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 49284.9us
I20260812 19:08:18.140501 2660 raft_consensus.cc:2781] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:18.212858 29569 raft_consensus.cc:1226] T d0dfbce6d2e3431ebc71db36ae455105 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260812 19:08:18.178715 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:18.231881 29338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 967) took 1565 ms. Trace:
I20260812 19:08:18.147754 2785 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:18.155895 2805 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:18.122933 2913 raft_consensus.cc:504] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.305016 29338 rpcz_store.cc:276] 0812 19:08:16.666363 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.984133 (+1317770us) service_pool.cc:227] Handling call
0812 19:08:18.045121 (+ 60988us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.231614 (+186493us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:18.231821 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:18.269539 29399 leader_election.cc:302] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:18.317900 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
W20260812 19:08:18.319262 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 49807us
I20260812 19:08:18.147722 2789 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:18.330425 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:18.238201 29326 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2353) took 1702 ms (client timeout 1931 ms). Trace:
I20260812 19:08:18.331605 29576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:18.332350 29576 raft_consensus.cc:2454] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:18.156234 2118 consensus_queue.cc:1046] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260812 19:08:18.326220 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:18.327744 29445 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
W20260812 19:08:18.334901 29326 rpcz_store.cc:269] 0812 19:08:16.535453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.871444 (+335991us) service_pool.cc:227] Handling call
0812 19:08:18.237966 (+1366522us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b54003d8454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:18.238128 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":229750528}
I20260812 19:08:18.367777 29578 raft_consensus.cc:2454] T c197fd0960e5419fafd6f7eec6f8959a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:18.245630 29528 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:18.369134 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 123607us
I20260812 19:08:18.370057 29528 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:18.232807 3022 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:18.049001 2611 consensus_queue.cc:260] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.316870 2660 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:18.384295 2660 raft_consensus.cc:3037] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:18.283393 2465 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260812 19:08:18.283237 1362 raft_consensus.cc:2781] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
I20260812 19:08:18.387746 29452 raft_consensus.cc:2379] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:18.317054 2904 raft_consensus.cc:482] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:18.318878 2805 raft_consensus.cc:482] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:18.155568 2968 raft_consensus.cc:504] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.395219 29281 leader_election.cc:302] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:18.402907 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffa1accbf0a749988850cc0f47a8efb0" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:18.403589 29578 raft_consensus.cc:2362] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 9. Re-sending same reply.
I20260812 19:08:18.403559 29281 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:18.162647 2905 raft_consensus.cc:2726] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:18.164726 2975 raft_consensus.cc:2781] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader election won for term 9
I20260812 19:08:18.327355 3021 raft_consensus.cc:482] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:18.140156 2926 raft_consensus.cc:504] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.382231 29445 raft_consensus.cc:1226] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:18.282752 2451 consensus_queue.cc:1035] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260812 19:08:18.377883 2284 raft_consensus.cc:2726] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260812 19:08:18.383339 2913 leader_election.cc:288] T 637fa5c7fa824870a0da61e2bc8400af P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.316845 2489 raft_consensus.cc:2726] T fd63b9965cdb4b568fd3b2860f8c2e61 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260812 19:08:18.387331 29337 raft_consensus.cc:2454] T 8d9f6c46c1a5494aa914b41b3a1616a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:08:18.283319 2786 raft_consensus.cc:2726] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:18.365907 29579 raft_consensus.cc:1226] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: []
I20260812 19:08:18.391105 29569 raft_consensus.cc:1226] T db322776a340462b91983d8e448cfe39 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
W20260812 19:08:18.392436 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 559416us
I20260812 19:08:18.209206 2596 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:18.358857 29399 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:18.565220 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 206466us
I20260812 19:08:18.566041 29399 leader_election.cc:302] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:18.165277 2645 raft_consensus.cc:2726] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:18.326625 2706 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.327194 2977 raft_consensus.cc:482] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:18.601672 30674 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.024s sys 0.020s
I20260812 19:08:18.207600 2657 raft_consensus.cc:2781] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader election won for term 7
W20260812 19:08:18.531265 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 127873us
I20260812 19:08:18.603155 29445 raft_consensus.cc:1226] T d05c973d4068462fa79a7bec57fc6e89 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:18.603466 29281 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:18.603224 30674 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.558s user 0.025s sys 0.020s
I20260812 19:08:18.377688 2785 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:18.526846 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:18.606451 29574 raft_consensus.cc:2396] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260812 19:08:18.534046 29567 raft_consensus.cc:1261] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260812 19:08:18.326881 2878 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.179s
I20260812 19:08:18.369462 2687 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:18.542809 29446 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate dd7e8c7a8bc54e73a39411cfc241fe9c in current term 9: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:18.553835 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:18.596616 30681 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s user 0.010s sys 0.032s
W20260812 19:08:18.629856 30681 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.603s user 0.011s sys 0.032s
I20260812 19:08:18.598097 29399 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:18.335278 2861 raft_consensus.cc:2726] T ffa1accbf0a749988850cc0f47a8efb0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:18.375041 2919 consensus_queue.cc:1035] T fd63b9965cdb4b568fd3b2860f8c2e61 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.184s
I20260812 19:08:18.401223 2789 raft_consensus.cc:686] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:18.540822 2358 raft_consensus.cc:2726] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260812 19:08:18.639631 29281 leader_election.cc:302] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:18.402338 2805 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:18.553412 1362 raft_consensus.cc:686] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:18.660727 30685 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s user 0.022s sys 0.027s
I20260812 19:08:18.671258 29453 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 7.
W20260812 19:08:18.566838 2959 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.009s sys 0.028s
I20260812 19:08:18.639751 2905 raft_consensus.cc:2781] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:18.526412 2792 raft_consensus.cc:2726] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260812 19:08:18.687556 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:18.538857 2822 raft_consensus.cc:482] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:18.688314 29445 raft_consensus.cc:3032] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Stepping down as leader of term 10
I20260812 19:08:18.400735 2782 consensus_queue.cc:1035] T da12498df45141cd80e5cb36e68775c5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.270s
I20260812 19:08:18.400904 3012 consensus_queue.cc:1035] T fe3191acdfe44a019be79eda30b9e79b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.240s
I20260812 19:08:18.494021 2465 consensus_queue.cc:1046] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.177s
W20260812 19:08:18.670042 30686 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.010s sys 0.008s
I20260812 19:08:18.603248 2913 raft_consensus.cc:2726] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:18.391897 3022 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:18.690603 2959 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.685s user 0.009s sys 0.033s
I20260812 19:08:18.726711 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 1012) took 2482 ms. Trace:
I20260812 19:08:18.727119 30670 rpcz_store.cc:276] 0812 19:08:16.244153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:16.888885 (+644732us) service_pool.cc:227] Handling call
0812 19:08:16.914343 (+ 25458us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.413666 (+499323us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:18.726475 (+1312809us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54005b1154. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:18.726648 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":854658944}
I20260812 19:08:18.568643 2975 raft_consensus.cc:686] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 LEADER]: Becoming Leader. State: Replica: dd7e8c7a8bc54e73a39411cfc241fe9c, State: Running, Role: LEADER
I20260812 19:08:18.537936 3011 consensus_queue.cc:1035] T ed9c55b662464bbd8e692e47a2f7cc2c P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.130s
I20260812 19:08:18.698385 29579 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:18.715267 29445 raft_consensus.cc:729] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:08:18.699230 30685 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.560s user 0.023s sys 0.027s
W20260812 19:08:18.718437 30686 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s user 0.010s sys 0.008s
I20260812 19:08:18.639262 2613 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P 6dd313c6592d4f33bab314a73f28b007 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260812 19:08:18.697496 2804 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s user 0.005s sys 0.007s
I20260812 19:08:18.375878 2283 consensus_queue.cc:260] T 647dfc8bad934787a924a06988dda474 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.495064 2904 raft_consensus.cc:504] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.596462 2284 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
W20260812 19:08:18.730062 30679 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s user 0.006s sys 0.010s
I20260812 19:08:18.697767 2905 raft_consensus.cc:482] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:18.541774 3021 raft_consensus.cc:504] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.747130 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
W20260812 19:08:18.748822 30679 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s user 0.007s sys 0.010s
I20260812 19:08:18.728911 2358 raft_consensus.cc:2781] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:18.673609 2977 raft_consensus.cc:504] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.732457 2959 raft_consensus.cc:686] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
W20260812 19:08:18.781920 29458 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s user 0.012s sys 0.012s
I20260812 19:08:18.670856 2687 raft_consensus.cc:504] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:18.798199 1986 consensus_meta.cc:287] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.689s user 0.009s sys 0.000s
W20260812 19:08:18.745797 2956 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.762s user 0.012s sys 0.030s
I20260812 19:08:18.494022 2879 consensus_queue.cc:260] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:18.746893 2804 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s user 0.005s sys 0.007s
W20260812 19:08:18.757579 2972 consensus_meta.cc:287] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.668s user 0.007s sys 0.003s
I20260812 19:08:18.792886 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c197fd0960e5419fafd6f7eec6f8959a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:18.776516 2905 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:18.776959 2284 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.722915 2822 raft_consensus.cc:504] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.782446 2358 raft_consensus.cc:482] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:18.690389 2789 consensus_queue.cc:237] T dfb5c8ad869c44379e8aa81d22549ae5 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:18.812263 29458 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s user 0.013s sys 0.012s
W20260812 19:08:18.820920 30623 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.587s user 0.009s sys 0.016s
W20260812 19:08:18.799096 30625 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.006s sys 0.014s
W20260812 19:08:18.833141 30625 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.548s user 0.006s sys 0.014s
W20260812 19:08:18.833628 30625 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 622744us
W20260812 19:08:18.822700 2956 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.845s user 0.012s sys 0.030s
I20260812 19:08:18.791865 1362 consensus_queue.cc:237] T db322776a340462b91983d8e448cfe39 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:18.827348 2625 consensus_meta.cc:287] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 0.724s user 0.009s sys 0.002s
I20260812 19:08:18.834367 30625 leader_election.cc:302] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:18.840973 30625 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:18.826217 2968 leader_election.cc:288] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:18.835029 30623 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.601s user 0.009s sys 0.016s
I20260812 19:08:18.817893 2451 consensus_queue.cc:1035] T e87477250ac44a318f7c0aef2d89fa36 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260812 19:08:18.789342 2861 consensus_queue.cc:1046] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260812 19:08:18.843935 30625 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007
W20260812 19:08:18.844925 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 675192us
I20260812 19:08:18.836617 2926 leader_election.cc:288] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.834822 2358 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:18.837603 2956 raft_consensus.cc:686] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:18.857980 2451 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260812 19:08:18.860229 2777 consensus_meta.cc:287] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df: Time spent flushing consensus metadata: real 1.148s user 0.007s sys 0.004s
I20260812 19:08:18.862160 2284 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:18.863021 3034 log.cc:924] Time spent T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260812 19:08:18.863724 2904 leader_election.cc:288] T 015555682f5f4cf293ca3718be163873 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:18.868474 2195 consensus_meta.cc:287] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.728s user 0.009s sys 0.000s
W20260812 19:08:18.869256 2974 consensus_meta.cc:287] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.869s user 0.006s sys 0.003s
I20260812 19:08:18.838521 2959 consensus_queue.cc:237] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.852030 1810 raft_consensus.cc:2781] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 FOLLOWER]: Leader election won for term 4
I20260812 19:08:18.822434 2975 consensus_queue.cc:237] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.828219 2972 raft_consensus.cc:504] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.857578 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:18.882248 29579 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:18.870460 3022 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.848574 2804 raft_consensus.cc:2781] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:18.883214 2804 raft_consensus.cc:482] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:18.851805 3046 raft_consensus.cc:2726] T 59710c65553f42188b78bc9bc0324542 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260812 19:08:18.853570 2625 consensus_queue.cc:1035] T 8cbee78e8845445eb3a4cbba1399e988 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260812 19:08:18.875854 2451 raft_consensus.cc:482] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
W20260812 19:08:18.876566 3036 log.cc:924] Time spent T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01: Append to log took a long time: real 0.262s user 0.001s sys 0.000s
W20260812 19:08:18.873721 2610 consensus_meta.cc:287] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.760s user 0.007s sys 0.007s
I20260812 19:08:18.848783 3037 consensus_queue.cc:1046] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.157s
I20260812 19:08:18.862210 2369 consensus_queue.cc:1035] T 79bd0e82dc2545e7867d77ada5cfb445 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260812 19:08:18.883579 2804 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:18.892195 2891 consensus_meta.cc:287] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.805s user 0.010s sys 0.000s
I20260812 19:08:18.849072 3035 consensus_queue.cc:1046] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.144s
I20260812 19:08:18.883639 2828 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:18.880018 2657 raft_consensus.cc:686] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Leader. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:18.884027 2687 leader_election.cc:288] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.874717 2777 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.880436 2974 raft_consensus.cc:504] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:18.936439 3042 log.cc:924] Time spent T fe3191acdfe44a019be79eda30b9e79b P a34c47dd29c14748a018d716ac6c39df: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
W20260812 19:08:18.937667 2660 consensus_meta.cc:287] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.553s user 0.007s sys 0.003s
I20260812 19:08:18.946663 2974 leader_election.cc:288] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 12 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.939098 2972 leader_election.cc:288] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.900923 3021 leader_election.cc:288] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.928306 2956 consensus_queue.cc:237] T 8987f70c28f749d8b5f5ef83f3349efd P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.942528 2905 raft_consensus.cc:504] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:18.926949 3035 consensus_queue.cc:1225] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260812 19:08:18.944626 2805 raft_consensus.cc:504] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.904855 2891 raft_consensus.cc:2726] T 97cc899036ba4e0d85662880b5e35132 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260812 19:08:18.956033 2805 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:18.945393 2777 leader_election.cc:288] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.924396 2977 leader_election.cc:288] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:18.957881 2862 consensus_meta.cc:287] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.841s user 0.010s sys 0.000s
I20260812 19:08:18.967159 2905 leader_election.cc:288] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.970407 2804 raft_consensus.cc:504] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.972702 2804 leader_election.cc:288] T f6dae5f1cee3498787a4a28802d59d16 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:18.825315 29339 raft_consensus.cc:595] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:19.015194 29339 raft_consensus.cc:603] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:19.015195 29335 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
W20260812 19:08:19.020641 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 422446us
W20260812 19:08:19.022993 29451 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.740s user 0.006s sys 0.022s
W20260812 19:08:19.023517 29451 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.740s user 0.006s sys 0.022s
I20260812 19:08:19.025259 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 891) took 1833 ms. Trace:
W20260812 19:08:19.025485 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 380456us
I20260812 19:08:19.027740 29572 raft_consensus.cc:1226] T 8cbee78e8845445eb3a4cbba1399e988 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 6. Current term is 7. Ops: []
W20260812 19:08:19.030455 29528 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.500s user 0.009s sys 0.018s
W20260812 19:08:19.030972 29528 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.535s user 0.009s sys 0.018s
W20260812 19:08:19.031414 29528 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 639329us
I20260812 19:08:19.034667 29459 raft_consensus.cc:1226] T b83150d7d32c400083ab43de4f67f607 P 182784ca1eb74135b8f16d8894fa4b01 [term 12 FOLLOWER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 11. Current term is 12. Ops: []
I20260812 19:08:19.036300 29403 leader_election.cc:302] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:19.038259 30623 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.044883 30671 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:19.046630 30679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1051) took 1303 ms. Trace:
W20260812 19:08:19.079849 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 440161us
W20260812 19:08:19.088531 29403 connection.cc:232] RPC call timeout handler was delayed by -1.0048213130794466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260812 19:08:19.088639 30685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1056) took 1182 ms. Trace:
I20260812 19:08:18.890543 1810 raft_consensus.cc:686] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Becoming Leader. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:19.089993 2610 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.091300 2687 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:18.941119 2657 consensus_queue.cc:237] T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.102598 1810 consensus_queue.cc:237] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.105746 2610 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:18.946823 2660 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.108345 29338 raft_consensus.cc:1226] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260812 19:08:19.089871 30686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1054) took 1306 ms. Trace:
I20260812 19:08:19.111063 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "637fa5c7fa824870a0da61e2bc8400af" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.112807 29530 leader_election.cc:302] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.117770 2284 consensus_queue.cc:1035] T 015555682f5f4cf293ca3718be163873 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260812 19:08:19.120534 29399 leader_election.cc:302] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.113296 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed56ce03a49908ad6f468e43b5402" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.125722 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:34276 (request call id 1020) took 1458 ms. Trace:
I20260812 19:08:19.126219 30671 rpcz_store.cc:276] 0812 19:08:17.667620 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.911046 (+243426us) service_pool.cc:227] Handling call
0812 19:08:19.120925 (+1209879us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54005c03d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.125621 (+ 4696us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2502259584}
I20260812 19:08:18.896999 2451 raft_consensus.cc:504] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.130970 2489 consensus_queue.cc:1035] T 64325c7581194fafac6e6e2b485fbdc2 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260812 19:08:19.132781 2956 raft_consensus.cc:2781] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:19.133319 2956 raft_consensus.cc:482] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.133697 2956 raft_consensus.cc:3037] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:19.120743 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.134758 29450 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 11.
I20260812 19:08:19.135679 2959 raft_consensus.cc:2726] T 0e4ba354bddd43ec97a890c3bb577330 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260812 19:08:19.136147 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 79145.4us
I20260812 19:08:19.137279 2465 consensus_queue.cc:1035] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260812 19:08:19.138252 2465 consensus_queue.cc:1225] T 2743b6275ccc4fbdbe5f71c667adee8b P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Peer 182784ca1eb74135b8f16d8894fa4b01 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260812 19:08:19.093070 30679 rpcz_store.cc:276] 0812 19:08:17.743437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.998092 (+254655us) service_pool.cc:227] Handling call
0812 19:08:17.998439 (+ 347us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:18.005858 (+ 7419us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.005889 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:18.006375 (+ 486us) log.cc:842] Serialized 26 byte log entry
0812 19:08:18.778405 (+772030us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:18.778444 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:18.778517 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.046003 (+267486us) raft_consensus.cc:1671] finished
0812 19:08:19.046039 (+ 36us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.046342 (+ 303us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6809,"raft.queue_time_us":7010,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":6871,"thread_start_us":771718,"threads_started":1,"wal-append.queue_time_us":873423}
I20260812 19:08:19.136024 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
W20260812 19:08:19.150385 29529 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.589s user 0.011s sys 0.002s
W20260812 19:08:19.150918 29529 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.623s user 0.011s sys 0.002s
I20260812 19:08:19.026273 29451 rpcz_store.cc:276] 0812 19:08:17.192089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.826663 (+634574us) service_pool.cc:227] Handling call
0812 19:08:17.826987 (+ 324us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:17.885223 (+ 58236us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.885258 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:17.894571 (+ 9313us) log.cc:842] Serialized 26 byte log entry
0812 19:08:18.163697 (+269126us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:19.024167 (+860470us) log.cc:842] Serialized 14 byte log entry
0812 19:08:19.024674 (+ 507us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.024809 (+ 135us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.024877 (+ 68us) raft_consensus.cc:1671] finished
0812 19:08:19.024888 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.025111 (+ 223us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":798977,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":35259,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":31674,"thread_start_us":974015,"threads_started":2,"wal-append.queue_time_us":663351}
I20260812 19:08:19.117424 2782 consensus_queue.cc:1035] T 39e2c7b7710843cf96e72ac710028fa9 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260812 19:08:19.170421 2782 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.171214 2782 raft_consensus.cc:504] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.155077 2660 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.178757 2777 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.179710 2777 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.180372 2704 raft_consensus.cc:2726] T bf300c57bddf44ffb79e424049fe4c52 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:19.093799 30685 rpcz_store.cc:276] 0812 19:08:17.906592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.053513 (+146921us) service_pool.cc:227] Handling call
0812 19:08:18.053824 (+ 311us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.054076 (+ 252us) raft_consensus.cc:1520] Early marking committed up to index 8
0812 19:08:18.742984 (+688908us) log.cc:842] Serialized 14 byte log entry
0812 19:08:18.895985 (+153001us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:18.896232 (+ 247us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:18.896354 (+ 122us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.088136 (+191782us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.088452 (+ 316us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9085,"prepare.queue_time_us":768156,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":1821,"raft.queue_time_us":43429,"spinlock_wait_cycles":15894400,"thread_start_us":750389,"threads_started":2,"wal-append.queue_time_us":224727}
I20260812 19:08:19.157627 29450 raft_consensus.cc:1226] T d28ed56ce03a49908ad6f468e43b5402 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260812 19:08:19.159449 29459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39332 (request call id 2588) took 1752 ms. Trace:
I20260812 19:08:19.160969 30676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 953) took 1589 ms. Trace:
I20260812 19:08:19.162432 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:19.162787 30675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34288 (request call id 950) took 1595 ms. Trace:
I20260812 19:08:19.164026 30684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 1018) took 1496 ms. Trace:
I20260812 19:08:19.166548 29399 leader_election.cc:302] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.101258 2687 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.110311 30686 rpcz_store.cc:276] 0812 19:08:17.782978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.001366 (+218388us) service_pool.cc:227] Handling call
0812 19:08:18.019039 (+ 17673us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:08:18.019444 (+ 405us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.019476 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:08:18.020009 (+ 533us) log.cc:842] Serialized 52 byte log entry
0812 19:08:18.740408 (+720399us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:18.740452 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:18.740540 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.089276 (+348736us) raft_consensus.cc:1671] finished
0812 19:08:19.089315 (+ 39us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.089651 (+ 336us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2525,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":720075,"threads_started":1,"wal-append.queue_time_us":781404}
I20260812 19:08:19.168273 2786 raft_consensus.cc:2781] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:19.182816 30685 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260812 19:08:19.190315 30681 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34276 (request call id 1025) took 1495 ms. Trace:
I20260812 19:08:19.187230 2786 raft_consensus.cc:482] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.200685 2660 raft_consensus.cc:482] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.204720 2786 raft_consensus.cc:3037] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:19.205432 2660 raft_consensus.cc:504] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:19.206820 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 59153.5us
I20260812 19:08:19.207871 2876 leader_election.cc:288] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.208642 2660 leader_election.cc:288] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.181267 2710 raft_consensus.cc:3032] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Stepping down as leader of term 7
I20260812 19:08:19.193277 30679 raft_consensus.cc:1226] T 647dfc8bad934787a924a06988dda474 P a34c47dd29c14748a018d716ac6c39df [term 10 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:19.209450 2710 raft_consensus.cc:729] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:19.210973 2710 consensus_queue.cc:260] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.211769 2879 raft_consensus.cc:3037] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.212438 2710 raft_consensus.cc:3037] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.195232 2791 raft_consensus.cc:3032] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Stepping down as leader of term 8
I20260812 19:08:19.213624 2791 raft_consensus.cc:729] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a34c47dd29c14748a018d716ac6c39df, State: Running, Role: LEADER
I20260812 19:08:19.173888 2782 leader_election.cc:288] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.191525 2687 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.193504 2777 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.198485 2954 raft_consensus.cc:2726] T 8d9f6c46c1a5494aa914b41b3a1616a1 P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260812 19:08:19.200136 30623 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 162213us
I20260812 19:08:19.183684 30676 rpcz_store.cc:276] 0812 19:08:17.571804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.571916 (+ 112us) service_pool.cc:227] Handling call
0812 19:08:17.610735 (+ 38819us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.768618 (+157883us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.160342 (+1391724us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54003502d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.160779 (+ 437us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":312206720}
I20260812 19:08:19.219530 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2313) took 1246 ms. Trace:
I20260812 19:08:19.184075 30684 rpcz_store.cc:276] 0812 19:08:17.667115 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.946852 (+279737us) service_pool.cc:227] Handling call
0812 19:08:18.047169 (+100317us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.137514 (+ 90345us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.163620 (+1026106us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b540019fd50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.163842 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":230025600}
I20260812 19:08:19.183564 29459 rpcz_store.cc:276] 0812 19:08:17.406826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.986775 (+579949us) service_pool.cc:227] Handling call
0812 19:08:18.026418 (+ 39643us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.035324 (+1008906us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.159091 (+123767us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003f4654. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.159376 (+ 285us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2317199104}
I20260812 19:08:19.223830 29450 rpcz_store.cc:276] 0812 19:08:17.942856 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:19.156557 (+1213701us) service_pool.cc:227] Handling call
0812 19:08:19.157226 (+ 669us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:19.188612 (+ 31386us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.188847 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:19.204538 30681 rpcz_store.cc:276] 0812 19:08:17.694834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.987538 (+292704us) service_pool.cc:227] Handling call
0812 19:08:17.987902 (+ 364us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:17.988178 (+ 276us) raft_consensus.cc:1520] Early marking committed up to index 8
0812 19:08:18.630276 (+642098us) log.cc:842] Serialized 14 byte log entry
0812 19:08:18.850076 (+219800us) log.cc:842] Serialized 14 byte log entry
0812 19:08:19.179746 (+329670us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:19.179849 (+ 103us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:19.179985 (+ 136us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.183698 (+ 3713us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.190096 (+ 6398us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":808838,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":5429,"raft.queue_time_us":31050,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":178,"thread_start_us":1151618,"threads_started":3,"wal-append.queue_time_us":268429}
I20260812 19:08:19.227469 2704 consensus_queue.cc:1035] T c0dc7d63c9244e56b10c73fbe2608457 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260812 19:08:19.232537 29403 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.26.205.65:44295 timed out after 1.939s (SENT) [suppressed 62 similar messages]
I20260812 19:08:19.233194 2954 raft_consensus.cc:482] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:19.214154 2891 raft_consensus.cc:482] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.239638 30671 raft_consensus.cc:1226] T 9ebf3eea25534182aa1c9f148898dc8b P a34c47dd29c14748a018d716ac6c39df [term 4 LEADER]: Rejecting Update request from peer dd7e8c7a8bc54e73a39411cfc241fe9c for earlier term 3. Current term is 4. Ops: []
I20260812 19:08:19.239720 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf300c57bddf44ffb79e424049fe4c52" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.242072 2610 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.252219 2822 leader_election.cc:288] T 1226d05dc64043078e42cf470348655d P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
W20260812 19:08:19.252559 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 86365.7us
I20260812 19:08:19.253364 30623 leader_election.cc:302] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:19.256006 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b83150d7d32c400083ab43de4f67f607" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.251147 29281 leader_election.cc:302] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:19.184005 30675 rpcz_store.cc:276] 0812 19:08:17.566954 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.567134 (+ 180us) service_pool.cc:227] Handling call
0812 19:08:17.598343 (+ 31209us) raft_consensus.cc:1457] Updating replica for 2 ops
0812 19:08:17.712498 (+114155us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:17.712531 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0812 19:08:17.743194 (+ 30663us) log.cc:842] Serialized 52 byte log entry
0812 19:08:18.234680 (+491486us) raft_consensus.cc:1631] Marking committed up to 10
0812 19:08:18.812584 (+577904us) log.cc:842] Serialized 14 byte log entry
0812 19:08:18.815432 (+ 2848us) log.cc:842] Serialized 14 byte log entry
0812 19:08:18.815663 (+ 231us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.162033 (+346370us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002fb554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.162099 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.162293 (+ 194us) raft_consensus.cc:1671] finished
0812 19:08:19.162307 (+ 14us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.162611 (+ 304us) inbound_call.cc:173] Queueing success response
0812 19:08:19.176499 (+ 13888us) spinlock_profiling.cc:243] Waited 2.53 ms on lock 0x7b48000b04d8. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6ecb57f1 00007f9c6ecac8e7 00007f9c6ecac77a 00007f9c6eda095f 00007f9c6eda0627 00007f9c6edb4b2c 00007f9c6edb4ad8 00007f9c6edb4a58 00007f9c6edb4a13 00007f9c6edb3cbc 00007f9c6dd35be6
Metrics: {"mutex_wait_us":9854,"prepare.queue_time_us":624749,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":2240,"raft.queue_time_us":4991,"raft.run_cpu_time_us":3225,"raft.run_wall_time_us":397275,"spinlock_wait_cycles":272915584,"thread_start_us":909782,"threads_started":2,"wal-append.queue_time_us":937640}
I20260812 19:08:19.215142 2791 consensus_queue.cc:260] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.265868 29459 raft_consensus.cc:1261] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260812 19:08:19.264114 3035 raft_consensus.cc:2726] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:19.267817 29341 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41568 (request call id 979) took 1857 ms. Trace:
I20260812 19:08:19.268296 29341 rpcz_store.cc:276] 0812 19:08:17.410705 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.030052 (+619347us) service_pool.cc:227] Handling call
0812 19:08:18.030358 (+ 306us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.597331 (+566973us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.597363 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:18.597402 (+ 39us) raft_consensus.cc:1631] Marking committed up to 10
0812 19:08:18.597444 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.267287 (+669843us) spinlock_profiling.cc:243] Waited 567 ms on lock 0x7b5400181354. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.267367 (+ 80us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.267614 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1303445888}
I20260812 19:08:19.259884 2891 raft_consensus.cc:504] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.278198 2594 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.278942 2594 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.280855 2891 leader_election.cc:288] T 4737efcfbbb9484781f7335f47b8c2ae P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.281117 2594 leader_election.cc:288] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.281742 29450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2315) took 1159 ms. Trace:
I20260812 19:08:19.282188 29450 rpcz_store.cc:276] 0812 19:08:18.122218 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:19.243023 (+1120805us) service_pool.cc:227] Handling call
0812 19:08:19.255842 (+ 12819us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.274402 (+ 18560us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:19.274432 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:19.281412 (+ 6980us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:19.281471 (+ 59us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.281532 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.281652 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260812 19:08:19.284907 2791 raft_consensus.cc:3037] T 69508446a398466a943e7ad290aa8e0c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:19.285171 30674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1058) took 1349 ms. Trace:
I20260812 19:08:19.285547 30674 rpcz_store.cc:276] 0812 19:08:17.935857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.882837 (+946980us) service_pool.cc:227] Handling call
0812 19:08:18.883183 (+ 346us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.134249 (+251066us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:19.134304 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:19.134375 (+ 71us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:19.134443 (+ 68us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.284776 (+150333us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54004605d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.284826 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.284987 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":576051072}
I20260812 19:08:19.302268 2610 raft_consensus.cc:2781] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:19.302835 2610 raft_consensus.cc:482] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.303285 2610 raft_consensus.cc:3037] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.273900 2954 raft_consensus.cc:504] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.306619 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.305891 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.308288 2899 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.309038 2899 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.309999 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.311503 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.312105 29337 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 7.
I20260812 19:08:19.256790 2905 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.314934 29579 raft_consensus.cc:1261] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260812 19:08:19.317981 29458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41204 (request call id 902) took 2089 ms. Trace:
W20260812 19:08:19.317529 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 63709us
I20260812 19:08:19.318377 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.333442 29458 rpcz_store.cc:276] 0812 19:08:17.228955 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.977307 (+748352us) service_pool.cc:227] Handling call
0812 19:08:18.017513 (+ 40206us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:18.141821 (+124308us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.141856 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:18.142307 (+ 451us) log.cc:842] Serialized 26 byte log entry
0812 19:08:18.833210 (+690903us) raft_consensus.cc:1631] Marking committed up to 7
0812 19:08:19.245759 (+412549us) log.cc:842] Serialized 14 byte log entry
0812 19:08:19.317415 (+ 71656us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.317535 (+ 120us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.317606 (+ 71us) raft_consensus.cc:1671] finished
0812 19:08:19.317616 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.317774 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":133680,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":31436,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":12019,"thread_start_us":1073773,"threads_started":2,"wal-append.queue_time_us":735288}
I20260812 19:08:19.335641 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:19.334498 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5640e3977714f6e8ffeb9e65d7c70f0" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.333077 29579 pending_rounds.cc:76] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting all ops after (but not including) 2
I20260812 19:08:19.313798 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14e182b70d344161a275142a344cd1ba" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.318172 30670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1057) took 1353 ms. Trace:
W20260812 19:08:19.317339 2817 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.769s user 0.008s sys 0.021s
I20260812 19:08:19.316708 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37fc905b65e94f9c9ad9b1bdc4f62986" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:19.338125 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 105652us
I20260812 19:08:19.338137 29337 raft_consensus.cc:2379] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 182784ca1eb74135b8f16d8894fa4b01 in current term 9: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:19.313084 29281 leader_election.cc:302] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.315573 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:19.339455 30670 rpcz_store.cc:276] 0812 19:08:17.923180 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.830645 (+907465us) service_pool.cc:227] Handling call
0812 19:08:18.830962 (+ 317us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.831185 (+ 223us) raft_consensus.cc:1520] Early marking committed up to index 8
0812 19:08:18.841037 (+ 9852us) log.cc:842] Serialized 14 byte log entry
0812 19:08:19.225052 (+384015us) log.cc:842] Serialized 14 byte log entry
0812 19:08:19.244191 (+ 19139us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:19.246458 (+ 2267us) raft_consensus.cc:1631] Marking committed up to 8
0812 19:08:19.250602 (+ 4144us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.256296 (+ 5694us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.276187 (+ 19891us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":46259,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":104,"raft.queue_time_us":82472,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":1824,"thread_start_us":229140,"threads_started":2,"wal-append.queue_time_us":98499}
I20260812 19:08:19.341648 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:19.342193 2899 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:19.350947 29453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2595) took 1898 ms (client timeout 1752 ms). Trace:
I20260812 19:08:19.317339 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0dc7d63c9244e56b10c73fbe2608457" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.351792 30629 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:19.355005 2905 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.354996 30684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cbee78e8845445eb3a4cbba1399e988" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:19.295586 2822 raft_consensus.cc:482] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.356523 3046 raft_consensus.cc:3032] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 LEADER]: Stepping down as leader of term 9
I20260812 19:08:19.359333 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6469507c6875443ebd9d4c40bb47a54a" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.297755 1810 raft_consensus.cc:2781] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:19.317994 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5616609c9dad428f93e5627a7ca84f2a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.338646 29579 pending_rounds.cc:97] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260812 19:08:19.366719 29580 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:19.360764 2817 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.893s user 0.010s sys 0.023s
W20260812 19:08:19.364280 29453 rpcz_store.cc:269] 0812 19:08:17.452520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.536065 (+ 83545us) service_pool.cc:227] Handling call
0812 19:08:19.339246 (+1803181us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540006ccd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.350705 (+ 11459us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1997133056}
I20260812 19:08:19.370400 30674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79bd0e82dc2545e7867d77ada5cfb445" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.372957 29579 raft_consensus.cc:2866] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:19.373191 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.376708 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.379381 2878 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:19.379673 29341 raft_consensus.cc:2354] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8.
I20260812 19:08:19.380013 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.380859 29574 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:19.379436 29569 raft_consensus.cc:2379] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:19.378625 29337 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:19.367174 3046 raft_consensus.cc:729] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:19.369690 1810 raft_consensus.cc:482] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.391902 30679 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:19.392560 29578 raft_consensus.cc:2454] T 37fc905b65e94f9c9ad9b1bdc4f62986 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 12.
I20260812 19:08:19.367430 2954 leader_election.cc:288] T 514939f267f2450faf798117d94726ab P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.391083 29567 raft_consensus.cc:2454] T c0dc7d63c9244e56b10c73fbe2608457 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:19.381713 2878 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.408846 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
W20260812 19:08:19.409873 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 220410us
I20260812 19:08:19.411711 30675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.398209 1810 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.410380 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7229b6057eba4aae9e2485a7bf1fcf17" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:19.407965 29569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.417908 2899 raft_consensus.cc:482] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:19.420343 2968 raft_consensus.cc:3032] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:19.420748 2968 raft_consensus.cc:729] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:19.419981 2899 raft_consensus.cc:504] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.414474 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:19.425197 29567 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.431704 30679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0adb15c8e8a8443a9c36474d5714f204" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.434134 2899 leader_election.cc:288] T 6d8ba27ebf754ac098699f3efab22620 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.436396 2876 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.437059 2876 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.413684 2878 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:19.438230 2874 raft_consensus.cc:482] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 182784ca1eb74135b8f16d8894fa4b01)
I20260812 19:08:19.438905 2874 raft_consensus.cc:504] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.441100 2874 leader_election.cc:288] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.442915 29335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41568 (request call id 989) took 1307 ms. Trace:
W20260812 19:08:19.427996 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 64318.9us
I20260812 19:08:19.443334 29335 rpcz_store.cc:276] 0812 19:08:18.135244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.141527 (+ 6283us) service_pool.cc:227] Handling call
0812 19:08:19.442732 (+1301205us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54003d7a54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.442842 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1972648320}
I20260812 19:08:19.451790 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.452620 29453 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:19.453392 29335 raft_consensus.cc:1226] T 69508446a398466a943e7ad290aa8e0c P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:19.453653 29458 raft_consensus.cc:1226] T 2ea822fc3c0f456ab7c3f8d71f84ff27 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260812 19:08:19.455260 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.456882 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.458030 29572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 950) took 1265 ms. Trace:
I20260812 19:08:19.458207 29578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87477250ac44a318f7c0aef2d89fa36" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.458485 29572 rpcz_store.cc:276] 0812 19:08:18.192650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.233812 (+ 41162us) service_pool.cc:227] Handling call
0812 19:08:18.333671 (+ 99859us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.028372 (+694701us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.457749 (+429377us) spinlock_profiling.cc:243] Waited 693 ms on lock 0x7b54002003d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.457928 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1594497280}
I20260812 19:08:19.459170 29569 raft_consensus.cc:2454] T 8987f70c28f749d8b5f5ef83f3349efd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
W20260812 19:08:19.462474 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 149558us
I20260812 19:08:19.463189 2611 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 4 FOLLOWER]: Advancing to term 5
I20260812 19:08:19.457767 29459 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:19.468393 29574 raft_consensus.cc:2379] T 0adb15c8e8a8443a9c36474d5714f204 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 7: Already voted for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in this term.
I20260812 19:08:19.422418 2657 raft_consensus.cc:2726] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260812 19:08:19.456766 2358 consensus_meta.cc:287] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007: Time spent flushing consensus metadata: real 0.584s user 0.009s sys 0.004s
I20260812 19:08:19.443310 2876 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.422245 2861 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.488417 30671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34302 (request call id 1059) took 1355 ms. Trace:
I20260812 19:08:19.488859 30671 rpcz_store.cc:276] 0812 19:08:18.133510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:19.127733 (+994223us) service_pool.cc:227] Handling call
0812 19:08:19.128406 (+ 673us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.240439 (+112033us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.486029 (+245590us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54007526d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.488344 (+ 2315us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":255341312}
I20260812 19:08:19.490509 30629 leader_election.cc:398] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8.
I20260812 19:08:19.496572 30629 leader_election.cc:401] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260812 19:08:19.414764 2822 raft_consensus.cc:504] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.503820 30681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5c317250947485cab086b4bfab851bd" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df"
I20260812 19:08:19.504536 30681 raft_consensus.cc:3037] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.510051 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.511142 29334 raft_consensus.cc:2379] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 10: Already voted for candidate 6dd313c6592d4f33bab314a73f28b007 in this term.
I20260812 19:08:19.518641 29282 leader_election.cc:302] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
W20260812 19:08:19.520411 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 76297.3us
I20260812 19:08:19.524323 30629 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:19.523427 29339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:41158 (request call id 30) took 1390 ms. Trace:
I20260812 19:08:19.506557 2777 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.529966 2777 raft_consensus.cc:504] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.531458 2861 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.532167 2777 leader_election.cc:288] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.532891 2968 consensus_queue.cc:260] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.534502 2968 raft_consensus.cc:3037] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:19.534646 29527 leader_election.cc:302] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.535671 2778 raft_consensus.cc:3037] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.537050 2610 raft_consensus.cc:504] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.527908 29341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.538322 29575 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 951) took 1338 ms. Trace:
I20260812 19:08:19.529939 3095 consensus_queue.cc:1035] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260812 19:08:19.539568 2822 leader_election.cc:288] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.539921 29572 raft_consensus.cc:1261] T ffa1accbf0a749988850cc0f47a8efb0 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260812 19:08:19.511270 2905 leader_election.cc:288] T ed9c55b662464bbd8e692e47a2f7cc2c P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.548676 29576 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 955) took 1182 ms. Trace:
I20260812 19:08:19.549535 2465 raft_consensus.cc:2726] T 6469507c6875443ebd9d4c40bb47a54a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 19:08:19.541838 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4737efcfbbb9484781f7335f47b8c2ae" candidate_uuid: "182784ca1eb74135b8f16d8894fa4b01" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.546305 3027 raft_consensus.cc:2781] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 19:08:19.528403 29399 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.492988 2358 raft_consensus.cc:504] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.543440 29575 rpcz_store.cc:276] 0812 19:08:18.202284 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.203264 (+ 980us) service_pool.cc:227] Handling call
0812 19:08:18.327119 (+123855us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:18.327524 (+ 405us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.327552 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:18.341995 (+ 14443us) log.cc:842] Serialized 26 byte log entry
0812 19:08:19.139757 (+797762us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b180048a240. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c6fd858b1 00007f9c6fd92661 00007f9c66ed4345 00007f9c6fcffa39 00007f9c66ef18bb 00007f9c66eefcbf 00007f9c66ef1e22 00007f9c69a8f5bb 00007f9c69a95063 00007f9c69a95411 00007f9c69de25cf
0812 19:08:19.337907 (+198150us) raft_consensus.cc:1631] Marking committed up to 9
0812 19:08:19.337941 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.338051 (+ 110us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.537734 (+199683us) raft_consensus.cc:1671] finished
0812 19:08:19.537757 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.538104 (+ 347us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":6482,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11354,"spinlock_wait_cycles":70432128,"thread_start_us":197743,"threads_started":1,"wal-append.queue_time_us":1169474}
I20260812 19:08:19.549055 29576 rpcz_store.cc:276] 0812 19:08:18.339225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.344786 (+ 5561us) service_pool.cc:227] Handling call
0812 19:08:18.345147 (+ 361us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.800167 (+455020us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.800199 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:18.800239 (+ 40us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:18.800284 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.506151 (+705867us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b5400171e54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.512464 (+ 6313us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.521445 (+ 8981us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1045958400}
I20260812 19:08:19.527194 29339 rpcz_store.cc:276] 0812 19:08:18.121832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.143907 (+ 22075us) service_pool.cc:227] Handling call
0812 19:08:19.507380 (+1363473us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540039bcd4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e08bc3 00007f9c6ed5281a 00007f9c68451d04 00007f9c68451c92 00007f9c68451bc1 00007f9c68451b3c 00007f9c68450df2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.512320 (+ 4940us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1565335808}
I20260812 19:08:19.547668 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59710c65553f42188b78bc9bc0324542" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.533622 2861 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.556141 2358 leader_election.cc:288] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.525831 3046 consensus_queue.cc:260] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:19.558557 3117 log.cc:924] Time spent T d0dfbce6d2e3431ebc71db36ae455105 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.155s user 0.001s sys 0.000s
I20260812 19:08:19.558970 3022 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.561592 29335 raft_consensus.cc:1226] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer a34c47dd29c14748a018d716ac6c39df for earlier term 5. Current term is 6. Ops: []
I20260812 19:08:19.562297 29456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2292) took 2449 ms. Trace:
I20260812 19:08:19.562713 29456 rpcz_store.cc:276] 0812 19:08:17.112762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.117605 (+ 4843us) service_pool.cc:227] Handling call
0812 19:08:17.136799 (+ 19194us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.097733 (+960934us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.561784 (+1464051us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54000c67d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:19.562104 (+ 320us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2208538496}
I20260812 19:08:19.563978 29339 raft_consensus.cc:1226] T 4d3af04408974eeab695361c3b2a8410 P 6dd313c6592d4f33bab314a73f28b007 [term 11 LEADER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260812 19:08:19.557799 3046 raft_consensus.cc:3037] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Advancing to term 10
I20260812 19:08:19.565840 29339 raft_consensus.cc:1226] T 514939f267f2450faf798117d94726ab P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 9. Current term is 10. Ops: []
I20260812 19:08:19.568552 2891 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:19.569625 29281 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:19.571587 29282 leader_election.cc:302] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:19.572552 2687 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260812 19:08:19.573432 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 129149us
W20260812 19:08:19.558863 3078 log.cc:924] Time spent T 39e2c7b7710843cf96e72ac710028fa9 P 6dd313c6592d4f33bab314a73f28b007: Append to log took a long time: real 0.372s user 0.003s sys 0.000s
W20260812 19:08:19.624876 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 110431us
W20260812 19:08:19.572685 29446 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39332 (request call id 2590) took 2139 ms (client timeout 1857 ms). Trace:
W20260812 19:08:19.626513 29446 rpcz_store.cc:269] 0812 19:08:17.432770 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.592403 (+159633us) service_pool.cc:227] Handling call
0812 19:08:19.572246 (+1979843us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54001aca54. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.572490 (+ 244us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1920373504}
I20260812 19:08:19.551918 3027 raft_consensus.cc:482] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:08:19.632050 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 57952.2us
I20260812 19:08:19.629982 29334 raft_consensus.cc:2454] T 59710c65553f42188b78bc9bc0324542 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 9.
I20260812 19:08:19.634738 29281 leader_election.cc:302] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.630235 2874 raft_consensus.cc:2781] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:19.629861 2879 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:19.644711 29457 raft_consensus.cc:1226] T a5640e3977714f6e8ffeb9e65d7c70f0 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Rejecting Update request from peer 6dd313c6592d4f33bab314a73f28b007 for earlier term 8. Current term is 9. Ops: []
I20260812 19:08:19.649899 29338 raft_consensus.cc:1261] T a5640e3977714f6e8ffeb9e65d7c70f0 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Refusing update from remote peer dd7e8c7a8bc54e73a39411cfc241fe9c: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260812 19:08:19.638387 3027 raft_consensus.cc:3037] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 12 FOLLOWER]: Advancing to term 13
W20260812 19:08:19.674845 29174 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55520 (request call id 376) took 19682 ms (client timeout 15000 ms). Trace:
W20260812 19:08:19.677647 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 155630us
W20260812 19:08:19.679080 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 44367.6us
W20260812 19:08:19.686146 29527 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 45915.6us
I20260812 19:08:19.631922 2611 consensus_queue.cc:1046] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260812 19:08:19.625705 3022 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.649200 2874 raft_consensus.cc:482] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.702373 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:19.692551 29567 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:19.677881 3053 raft_consensus.cc:2781] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:19.702700 29459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.701617 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.710578 29403 leader_election.cc:302] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c
W20260812 19:08:19.718766 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 84136.6us
I20260812 19:08:19.700995 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.648958 2879 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.647192 2358 raft_consensus.cc:482] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.706631 2822 raft_consensus.cc:482] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.735301 29580 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
I20260812 19:08:19.718887 2379 consensus_queue.cc:1046] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260812 19:08:19.744094 29281 leader_election.cc:302] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: 182784ca1eb74135b8f16d8894fa4b01, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.691164 2899 raft_consensus.cc:2726] T 14e182b70d344161a275142a344cd1ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 19:08:19.751804 29530 leader_election.cc:302] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:19.727998 2874 raft_consensus.cc:3037] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:19.728241 3053 raft_consensus.cc:482] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.711297 3035 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.742895 2879 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.756662 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:19.760891 3053 raft_consensus.cc:3037] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
W20260812 19:08:19.760171 29403 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 49814.6us
I20260812 19:08:19.761117 2379 raft_consensus.cc:3032] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:19.763106 29459 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:19.766783 30670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d8ba27ebf754ac098699f3efab22620" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.764844 29326 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2488) took 1066 ms. Trace:
I20260812 19:08:19.767707 29326 rpcz_store.cc:276] 0812 19:08:18.695314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.695463 (+ 149us) service_pool.cc:227] Handling call
0812 19:08:18.720862 (+ 25399us) raft_consensus.cc:1457] Updating replica for 1 ops
0812 19:08:18.725579 (+ 4717us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.725610 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0812 19:08:18.735069 (+ 9459us) log.cc:842] Serialized 26 byte log entry
0812 19:08:19.178422 (+443353us) raft_consensus.cc:1631] Marking committed up to 5
0812 19:08:19.178524 (+ 102us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.178688 (+ 164us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0812 19:08:19.558060 (+379372us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b44000232cc. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66e04a95 00007f9c66e054b2 00007f9c69aa1511 00007f9c69a9ce1a 00007f9c69a90bee 00007f9c69a8ff36 00007f9c69a8fb58 00007f9c69a99551 00007f9c69a99509 00007f9c69a99499
0812 19:08:19.746448 (+188388us) raft_consensus.cc:1671] finished
0812 19:08:19.746468 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.761343 (+ 14875us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4302,"raft.queue_time_us":4802,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12140,"spinlock_wait_cycles":68270336,"thread_start_us":355623,"threads_started":1,"wal-append.queue_time_us":398255}
I20260812 19:08:19.769217 2610 leader_election.cc:288] T c197fd0960e5419fafd6f7eec6f8959a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 election: Requested vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.767215 30671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a34c47dd29c14748a018d716ac6c39df" is_pre_election: true
I20260812 19:08:19.772166 30671 raft_consensus.cc:2454] T 69bfa848552945a18495c13ca6707d65 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 5.
I20260812 19:08:19.778205 2861 raft_consensus.cc:482] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.778477 2623 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.778944 2861 raft_consensus.cc:504] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.779145 2623 raft_consensus.cc:504] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.781085 2861 leader_election.cc:288] T 79bd0e82dc2545e7867d77ada5cfb445 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.764981 2379 raft_consensus.cc:729] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
W20260812 19:08:19.782572 2786 consensus_meta.cc:287] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c: Time spent flushing consensus metadata: real 0.577s user 0.009s sys 0.006s
I20260812 19:08:19.783026 2786 raft_consensus.cc:504] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.785151 2786 leader_election.cc:288] T 8d9f6c46c1a5494aa914b41b3a1616a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.785751 3011 raft_consensus.cc:2781] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 19:08:19.786218 3011 raft_consensus.cc:482] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6dd313c6592d4f33bab314a73f28b007)
I20260812 19:08:19.786391 2358 raft_consensus.cc:504] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.788781 2358 leader_election.cc:288] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.790515 2878 raft_consensus.cc:504] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.791061 1810 raft_consensus.cc:504] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.792757 2878 leader_election.cc:288] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 7 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.793284 1810 leader_election.cc:288] T bf312d0cb2a6414794d20e98e1edf8ba P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.793694 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6dae5f1cee3498787a4a28802d59d16" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01"
I20260812 19:08:19.761690 2791 raft_consensus.cc:482] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:19.794126 30679 raft_consensus.cc:2454] T 0adb15c8e8a8443a9c36474d5714f204 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:19.795490 29445 consensus_queue.cc:260] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.796697 29445 raft_consensus.cc:3037] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Advancing to term 11
I20260812 19:08:19.797762 30670 raft_consensus.cc:2379] T 6d8ba27ebf754ac098699f3efab22620 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6dd313c6592d4f33bab314a73f28b007 in current term 7: Already voted for candidate a34c47dd29c14748a018d716ac6c39df in this term.
I20260812 19:08:19.798427 29339 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8987f70c28f749d8b5f5ef83f3349efd"
dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
mode: GRACEFUL
new_leader_uuid: "182784ca1eb74135b8f16d8894fa4b01"
from {username='slave'} at 127.0.0.1:41158
I20260812 19:08:19.799001 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.799837 29446 raft_consensus.cc:2379] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df in current term 6: Already voted for candidate 182784ca1eb74135b8f16d8894fa4b01 in this term.
I20260812 19:08:19.800246 29451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2314) took 1856 ms. Trace:
I20260812 19:08:19.800661 29446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "514939f267f2450faf798117d94726ab" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.803387 29457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2304) took 2581 ms. Trace:
I20260812 19:08:19.803803 29457 rpcz_store.cc:276] 0812 19:08:17.221681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.971671 (+749990us) service_pool.cc:227] Handling call
0812 19:08:19.572936 (+1601265us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.680742 (+107806us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.803187 (+122445us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54001aca50. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.803327 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3681707264}
W20260812 19:08:19.804662 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 69983.1us
I20260812 19:08:19.805467 29281 leader_election.cc:302] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:19.801299 2822 raft_consensus.cc:504] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.794227 2612 raft_consensus.cc:2726] T 5616609c9dad428f93e5627a7ca84f2a P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:19.794414 29456 raft_consensus.cc:3037] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.800594 29451 rpcz_store.cc:276] 0812 19:08:17.943664 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:19.190715 (+1247051us) service_pool.cc:227] Handling call
0812 19:08:19.669135 (+478420us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.702562 (+ 33427us) raft_consensus.cc:1520] Early marking committed up to index 6
0812 19:08:19.702595 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:19.702666 (+ 71us) raft_consensus.cc:1631] Marking committed up to 6
0812 19:08:19.702711 (+ 45us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:19.702771 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:19.800044 (+ 97273us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54001ac550. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0da09 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.800168 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":7744,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":157,"spinlock_wait_cycles":1098965888}
I20260812 19:08:19.817852 29580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75e1d0973f2d43b6affb3b455c7789a1" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.817788 29567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e90268109844f62bf1e1ee425b2946a" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" is_pre_election: true
I20260812 19:08:19.808737 29282 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:19.796916 29453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ea5787113834ce596fc55df3da0e244" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.813270 30681 raft_consensus.cc:2454] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 8.
I20260812 19:08:19.824800 29326 raft_consensus.cc:1226] T 3cc924748e9a482e8536297e097a5226 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting Update request from peer 182784ca1eb74135b8f16d8894fa4b01 for earlier term 5. Current term is 6. Ops: []
W20260812 19:08:19.816757 30629 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 68610us
I20260812 19:08:19.772125 2891 raft_consensus.cc:2726] T 005defc807dc47d881add64226efb7ce P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260812 19:08:19.807036 29339 raft_consensus.cc:595] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Received request to transfer leadership to TS 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:19.831933 29339 raft_consensus.cc:603] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260812 19:08:19.817181 29399 leader_election.cc:302] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.830487 29453 raft_consensus.cc:2454] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:19.827353 29567 raft_consensus.cc:2454] T 6e90268109844f62bf1e1ee425b2946a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 6.
I20260812 19:08:19.833077 29446 raft_consensus.cc:2454] T 514939f267f2450faf798117d94726ab P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 10.
I20260812 19:08:19.814900 3011 raft_consensus.cc:3037] T 59710c65553f42188b78bc9bc0324542 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Advancing to term 9
I20260812 19:08:19.781311 2623 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.837613 29580 raft_consensus.cc:2454] T 75e1d0973f2d43b6affb3b455c7789a1 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 9.
I20260812 19:08:19.820567 2954 raft_consensus.cc:2726] T a5c317250947485cab086b4bfab851bd P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295) with higher term. Message: Invalid argument: T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a34c47dd29c14748a018d716ac6c39df for earlier term 7. Current term is 8.
I20260812 19:08:19.824723 2358 raft_consensus.cc:2781] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:19.842890 29579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cc924748e9a482e8536297e097a5226" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c"
I20260812 19:08:19.824404 2861 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.828948 2610 raft_consensus.cc:2781] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:19.848937 29342 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 8.
I20260812 19:08:19.853199 2804 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.853413 29451 raft_consensus.cc:1261] T 9ebf3eea25534182aa1c9f148898dc8b P 182784ca1eb74135b8f16d8894fa4b01 [term 4 FOLLOWER]: Refusing update from remote peer a34c47dd29c14748a018d716ac6c39df: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260812 19:08:19.846881 29399 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, a34c47dd29c14748a018d716ac6c39df; no voters: 6dd313c6592d4f33bab314a73f28b007
I20260812 19:08:19.853778 2804 raft_consensus.cc:504] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.855000 29451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb5c8ad869c44379e8aa81d22549ae5" candidate_uuid: "6dd313c6592d4f33bab314a73f28b007" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "182784ca1eb74135b8f16d8894fa4b01" is_pre_election: true
I20260812 19:08:19.855248 29399 leader_election.cc:302] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.855777 29451 raft_consensus.cc:2454] T dfb5c8ad869c44379e8aa81d22549ae5 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dd313c6592d4f33bab314a73f28b007 in term 7.
I20260812 19:08:19.856081 2804 leader_election.cc:288] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.856964 3064 raft_consensus.cc:482] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.857244 29451 raft_consensus.cc:1261] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260812 19:08:19.857633 2786 raft_consensus.cc:482] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.858322 2786 raft_consensus.cc:504] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } attrs { replace: true } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:19.685245 29174 rpcz_store.cc:269] 0812 19:07:59.952877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:07:59.953066 (+ 189us) service_pool.cc:227] Handling call
0812 19:08:19.629672 (+19676606us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0812 19:08:09.766548 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0812 19:08:09.850894 (+ 84346us) write_op.cc:432] Acquiring schema lock in shared mode
0812 19:08:09.850946 (+ 52us) write_op.cc:435] Acquired schema lock
0812 19:08:09.850962 (+ 16us) tablet.cc:660] Decoding operations
0812 19:08:09.851955 (+ 993us) write_op.cc:620] Acquiring the partition lock for write op
0812 19:08:09.852039 (+ 84us) write_op.cc:641] Partition lock acquired for write op
0812 19:08:09.852064 (+ 25us) tablet.cc:683] Acquiring locks for 10 operations
0812 19:08:09.852904 (+ 840us) tablet.cc:699] Row locks acquired
0812 19:08:09.852938 (+ 34us) write_op.cc:260] PREPARE: finished
0812 19:08:09.853104 (+ 166us) write_op.cc:270] Start()
0812 19:08:09.853233 (+ 129us) write_op.cc:276] Timestamp: P: 1786561689853042 usec, L: 0
0812 19:08:09.853266 (+ 33us) op_driver.cc:331] REPLICATION: starting
0812 19:08:09.853919 (+ 653us) log.cc:842] Serialized 3999 byte log entry
0812 19:08:10.360208 (+506289us) op_driver.cc:444] REPLICATION: finished
0812 19:08:10.489873 (+129665us) write_op.cc:301] APPLY: starting
0812 19:08:10.490011 (+ 138us) tablet.cc:1365] starting BulkCheckPresence
0812 19:08:10.490266 (+ 255us) tablet.cc:1368] finished BulkCheckPresence
0812 19:08:10.490381 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0812 19:08:10.562323 (+ 71942us) tablet.cc:1381] finished ApplyRowOperation cycle
0812 19:08:10.562659 (+ 336us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0812 19:08:10.562707 (+ 48us) write_op.cc:312] APPLY: finished
0812 19:08:10.563094 (+ 387us) log.cc:842] Serialized 77 byte log entry
0812 19:08:10.563292 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:10.563441 (+ 149us) write_op.cc:454] Released schema lock
0812 19:08:10.563504 (+ 63us) write_op.cc:341] FINISH: Updating metrics
0812 19:08:10.563865 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0812 19:08:10.563893 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1468432,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":117314,"apply.run_cpu_time_us":4339,"apply.run_wall_time_us":74387,"num_ops":10,"prepare.queue_time_us":248602,"prepare.run_cpu_time_us":30797,"prepare.run_wall_time_us":272426,"raft.queue_time_us":216496,"raft.run_cpu_time_us":10655,"raft.run_wall_time_us":168175,"replication_time_us":503371,"thread_start_us":516039,"threads_started":4,"wal-append.queue_time_us":135221,"wal-append.run_cpu_time_us":123433,"wal-append.run_wall_time_us":3191685}]]}
I20260812 19:08:19.856848 29399 leader_election.cc:302] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.846843 29281 leader_election.cc:302] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:19.842152 2891 raft_consensus.cc:2726] T 1226d05dc64043078e42cf470348655d P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:19.857664 29451 pending_rounds.cc:76] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 3
I20260812 19:08:19.870781 29451 pending_rounds.cc:97] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260812 19:08:19.871037 29451 raft_consensus.cc:2866] T d0dfbce6d2e3431ebc71db36ae455105 P 182784ca1eb74135b8f16d8894fa4b01 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:19.882186 3099 consensus_queue.cc:1035] T a5640e3977714f6e8ffeb9e65d7c70f0 P dd7e8c7a8bc54e73a39411cfc241fe9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260812 19:08:19.883602 29337 raft_consensus.cc:1261] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Refusing update from remote peer 182784ca1eb74135b8f16d8894fa4b01: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260812 19:08:19.884033 29337 pending_rounds.cc:76] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Aborting all ops after (but not including) 3
I20260812 19:08:19.884254 29337 pending_rounds.cc:97] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260812 19:08:19.884446 29281 leader_election.cc:302] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.884855 2710 consensus_queue.cc:1035] T 7adcd8cac12a4350a441ca5a35090ec9 P 6dd313c6592d4f33bab314a73f28b007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260812 19:08:19.782564 2379 consensus_queue.cc:260] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.886153 2899 raft_consensus.cc:2726] T 7229b6057eba4aae9e2485a7bf1fcf17 P 6dd313c6592d4f33bab314a73f28b007 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260812 19:08:19.887394 2789 raft_consensus.cc:482] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.887863 2789 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.823168 3035 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.890293 2379 raft_consensus.cc:3037] T bf300c57bddf44ffb79e424049fe4c52 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.891672 3035 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.892233 2905 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Leader election won for term 11
I20260812 19:08:19.881608 2451 leader_election.cc:288] T 2743b6275ccc4fbdbe5f71c667adee8b P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
W20260812 19:08:19.896004 3143 log.cc:924] Time spent T e87477250ac44a318f7c0aef2d89fa36 P dd7e8c7a8bc54e73a39411cfc241fe9c: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260812 19:08:19.902110 3064 raft_consensus.cc:504] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.904515 3064 leader_election.cc:288] T 8b7629c8adbd447596c2e821c5ac0a99 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.908888 2451 raft_consensus.cc:482] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.852631 2610 raft_consensus.cc:482] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:19.909600 2610 raft_consensus.cc:3037] T 69bfa848552945a18495c13ca6707d65 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Advancing to term 7
I20260812 19:08:19.909554 2451 raft_consensus.cc:504] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.910102 3056 raft_consensus.cc:482] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election (detected failure of leader dd7e8c7a8bc54e73a39411cfc241fe9c)
I20260812 19:08:19.910876 3056 raft_consensus.cc:504] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.912370 2451 leader_election.cc:288] T db322776a340462b91983d8e448cfe39 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.913822 3056 leader_election.cc:288] T 64325c7581194fafac6e6e2b485fbdc2 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.885659 29281 leader_election.cc:302] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:19.914685 2822 leader_election.cc:288] T 9ebf3eea25534182aa1c9f148898dc8b P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.832222 2791 raft_consensus.cc:504] T ed9c55b662464bbd8e692e47a2f7cc2c P a34c47dd29c14748a018d716ac6c39df [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:19.861672 2465 raft_consensus.cc:482] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.915349 2687 raft_consensus.cc:2781] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:19.917893 2786 leader_election.cc:288] T 005defc807dc47d881add64226efb7ce P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:19.918354 29334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf312d0cb2a6414794d20e98e1edf8ba" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.920456 29326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69bfa848552945a18495c13ca6707d65" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.921198 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97cc899036ba4e0d85662880b5e35132" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:19.967661 29334 raft_consensus.cc:3037] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:19.968158 29338 raft_consensus.cc:2454] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 5.
I20260812 19:08:19.851707 2861 raft_consensus.cc:504] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.856791 2707 consensus_queue.cc:1046] T 7229b6057eba4aae9e2485a7bf1fcf17 P a34c47dd29c14748a018d716ac6c39df [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
W20260812 19:08:19.922746 2956 consensus_meta.cc:287] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01: Time spent flushing consensus metadata: real 0.788s user 0.011s sys 0.005s
W20260812 19:08:19.975263 29530 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.65:44295, user_credentials={real_user=slave}} blocked reactor thread for 219957us
I20260812 19:08:19.925088 29577 raft_consensus.cc:2454] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182784ca1eb74135b8f16d8894fa4b01 in term 6.
I20260812 19:08:19.869294 3066 raft_consensus.cc:2726] T 8ea5787113834ce596fc55df3da0e244 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260812 19:08:19.850564 2358 raft_consensus.cc:482] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260812 19:08:19.934589 29342 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33260 (request call id 2387) took 2915 ms (client timeout 1939 ms). Trace:
W20260812 19:08:19.935285 29282 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.26.205.66:43659, user_credentials={real_user=slave}} blocked reactor thread for 69624.9us
W20260812 19:08:19.935556 29399 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.68:41925, user_credentials={real_user=slave}} blocked reactor thread for 78831.3us
I20260812 19:08:19.884467 29337 raft_consensus.cc:2866] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:19.930559 2879 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
W20260812 19:08:19.967746 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 82168.4us
I20260812 19:08:19.985426 29281 leader_election.cc:302] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.969280 30629 leader_election.cc:302] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
W20260812 19:08:19.969766 29452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39326 (request call id 2303) took 2534 ms (client timeout 2988 ms). Trace:
I20260812 19:08:19.889781 2789 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659)
I20260812 19:08:19.974737 29579 raft_consensus.cc:3037] T 3cc924748e9a482e8536297e097a5226 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:19.909641 3027 raft_consensus.cc:504] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:19.972877 3046 raft_consensus.cc:2726] T 69bfa848552945a18495c13ca6707d65 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 19:08:19.971639 2687 raft_consensus.cc:482] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:19.996078 29338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db322776a340462b91983d8e448cfe39" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
W20260812 19:08:19.980183 29342 rpcz_store.cc:269] 0812 19:08:17.019634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.203536 (+183902us) service_pool.cc:227] Handling call
0812 19:08:18.317736 (+1114200us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b5800003398. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c66d36f11 00007f9c66eba851 00007f9c66eba702 00007f9c6fd9eb09 00007f9c6ecda730 00007f9c6eda079b 00007f9c6eda0821 00007f9c6ed501b0 00007f9c6ed4fa94 00007f9c68443c04 00007f9c68443b92
0812 19:08:19.934310 (+1616574us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003903d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.934473 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2886697088}
I20260812 19:08:19.990617 29399 leader_election.cc:302] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: a34c47dd29c14748a018d716ac6c39df, dd7e8c7a8bc54e73a39411cfc241fe9c
I20260812 19:08:19.977093 29456 raft_consensus.cc:2454] T f6dae5f1cee3498787a4a28802d59d16 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 6.
W20260812 19:08:19.987365 29452 rpcz_store.cc:269] 0812 19:08:17.212609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.229227 (+ 16618us) service_pool.cc:227] Handling call
0812 19:08:19.644063 (+2414836us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001ac554. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:19.746535 (+102472us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2329853696}
I20260812 19:08:19.993929 29281 leader_election.cc:302] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters:
I20260812 19:08:19.976336 2284 raft_consensus.cc:482] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:19.976056 2905 raft_consensus.cc:686] T f55b37aa5102423ebbccef3f872d85e8 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 LEADER]: Becoming Leader. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:20.001858 29342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d9f6c46c1a5494aa914b41b3a1616a1" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007"
I20260812 19:08:19.992888 2465 raft_consensus.cc:504] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:20.005056 2465 leader_election.cc:288] T 7229b6057eba4aae9e2485a7bf1fcf17 P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:20.008661 29337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f55b37aa5102423ebbccef3f872d85e8" candidate_uuid: "a34c47dd29c14748a018d716ac6c39df" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:20.010413 29331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33256 (request call id 2484) took 1837 ms. Trace:
I20260812 19:08:20.010792 29331 rpcz_store.cc:276] 0812 19:08:18.173318 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.173476 (+ 158us) service_pool.cc:227] Handling call
0812 19:08:18.327647 (+154171us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:18.796792 (+469145us) raft_consensus.cc:1520] Early marking committed up to index 0
0812 19:08:18.796824 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0812 19:08:18.796866 (+ 42us) raft_consensus.cc:1631] Marking committed up to 4
0812 19:08:18.796909 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:20.010023 (+1213114us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b540011d0d4. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e0ef51 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612 00007f9c68441541 00007f9c684414bc 00007f9c68440772 00007f9c6824494c 00007f9c68243d86
0812 19:08:20.010092 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0812 19:08:20.010245 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1078449024}
I20260812 19:08:20.012127 29331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "005defc807dc47d881add64226efb7ce" candidate_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6dd313c6592d4f33bab314a73f28b007" is_pre_election: true
I20260812 19:08:20.012707 29331 raft_consensus.cc:2454] T 005defc807dc47d881add64226efb7ce P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd7e8c7a8bc54e73a39411cfc241fe9c in term 8.
I20260812 19:08:20.013844 2379 raft_consensus.cc:2781] T 8987f70c28f749d8b5f5ef83f3349efd P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 19:08:20.015859 3027 leader_election.cc:288] T 37fc905b65e94f9c9ad9b1bdc4f62986 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 13 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.984524 2358 raft_consensus.cc:3037] T 0adb15c8e8a8443a9c36474d5714f204 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Advancing to term 8
I20260812 19:08:20.017815 3066 consensus_queue.cc:1046] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260812 19:08:19.978737 2956 raft_consensus.cc:504] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:20.020517 2956 leader_election.cc:288] T 39e2c7b7710843cf96e72ac710028fa9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 9 election: Requested vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:19.993425 2804 raft_consensus.cc:2781] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 19:08:20.022423 2804 raft_consensus.cc:482] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:20.022825 2804 raft_consensus.cc:3037] T 97cc899036ba4e0d85662880b5e35132 P a34c47dd29c14748a018d716ac6c39df [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:20.022926 29399 leader_election.cc:302] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01; no voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df
I20260812 19:08:20.023208 3046 raft_consensus.cc:2726] T fe3191acdfe44a019be79eda30b9e79b P 182784ca1eb74135b8f16d8894fa4b01 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 19:08:20.024017 3046 raft_consensus.cc:2726] T 647dfc8bad934787a924a06988dda474 P 182784ca1eb74135b8f16d8894fa4b01 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260812 19:08:20.024338 3066 consensus_queue.cc:1225] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer 6dd313c6592d4f33bab314a73f28b007 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260812 19:08:20.029479 29445 raft_consensus.cc:1261] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: Refusing update from remote peer 6dd313c6592d4f33bab314a73f28b007: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260812 19:08:20.009495 29337 raft_consensus.cc:2454] T f55b37aa5102423ebbccef3f872d85e8 P 6dd313c6592d4f33bab314a73f28b007 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 11.
I20260812 19:08:20.030184 3046 raft_consensus.cc:3032] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Stepping down as leader of term 5
W20260812 19:08:20.034480 29281 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.26.205.67:44573, user_credentials={real_user=slave}} blocked reactor thread for 47842.8us
I20260812 19:08:20.035240 29282 leader_election.cc:302] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182784ca1eb74135b8f16d8894fa4b01, 6dd313c6592d4f33bab314a73f28b007; no voters:
I20260812 19:08:20.033980 30629 leader_election.cc:302] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, a34c47dd29c14748a018d716ac6c39df; no voters:
I20260812 19:08:20.029796 29445 pending_rounds.cc:76] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Aborting all ops after (but not including) 8
I20260812 19:08:20.000294 2687 raft_consensus.cc:3037] T c0dc7d63c9244e56b10c73fbe2608457 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:20.045941 29281 leader_election.cc:302] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6dd313c6592d4f33bab314a73f28b007, dd7e8c7a8bc54e73a39411cfc241fe9c; no voters: 182784ca1eb74135b8f16d8894fa4b01
I20260812 19:08:20.050282 3149 raft_consensus.cc:2726] T e87477250ac44a318f7c0aef2d89fa36 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 19:08:20.052510 29445 pending_rounds.cc:97] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260812 19:08:20.056398 29445 raft_consensus.cc:2866] T 4d3af04408974eeab695361c3b2a8410 P 182784ca1eb74135b8f16d8894fa4b01 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 19:08:20.040789 2876 raft_consensus.cc:2781] T a5c317250947485cab086b4bfab851bd P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election won for term 8
I20260812 19:08:20.039142 3046 raft_consensus.cc:729] T 3cc924748e9a482e8536297e097a5226 P 182784ca1eb74135b8f16d8894fa4b01 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:20.051966 2707 raft_consensus.cc:2781] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 19:08:20.066751 2707 raft_consensus.cc:482] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 19:08:20.067134 2707 raft_consensus.cc:3037] T f55b37aa5102423ebbccef3f872d85e8 P a34c47dd29c14748a018d716ac6c39df [term 11 FOLLOWER]: Advancing to term 12
I20260812 19:08:20.051036 2710 raft_consensus.cc:2781] T dfb5c8ad869c44379e8aa81d22549ae5 P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 19:08:20.074728 3022 leader_election.cc:288] T 97cc899036ba4e0d85662880b5e35132 P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:20.071048 2822 raft_consensus.cc:482] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a34c47dd29c14748a018d716ac6c39df)
I20260812 19:08:20.075770 2822 raft_consensus.cc:504] T 69508446a398466a943e7ad290aa8e0c P dd7e8c7a8bc54e73a39411cfc241fe9c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:20.076328 2874 raft_consensus.cc:504] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
W20260812 19:08:20.077174 29577 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57278 (request call id 2272) took 2246 ms (client timeout 1647 ms). Trace:
W20260812 19:08:20.077576 29577 rpcz_store.cc:269] 0812 19:08:17.831079 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:17.893063 (+ 61984us) service_pool.cc:227] Handling call
0812 19:08:20.076926 (+2183863us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b540012b454. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69e122ae 00007f9c6ed4fc27 00007f9c68443c04 00007f9c68443b92 00007f9c68443ac1 00007f9c68443a3c 00007f9c68442cf2 00007f9c6824494c 00007f9c68243d86 00007f9c6824372f
0812 19:08:20.077085 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4547975936}
I20260812 19:08:20.054746 3053 raft_consensus.cc:504] T 8ea5787113834ce596fc55df3da0e244 P a34c47dd29c14748a018d716ac6c39df [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:20.080235 2660 raft_consensus.cc:3032] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 19:08:20.080658 2660 raft_consensus.cc:729] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6dd313c6592d4f33bab314a73f28b007, State: Running, Role: LEADER
I20260812 19:08:20.081132 2899 raft_consensus.cc:482] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 19:08:20.063086 3149 raft_consensus.cc:2781] T 75e1d0973f2d43b6affb3b455c7789a1 P 6dd313c6592d4f33bab314a73f28b007 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 19:08:20.081801 2899 raft_consensus.cc:504] T d05c973d4068462fa79a7bec57fc6e89 P 6dd313c6592d4f33bab314a73f28b007 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:20.081617 2660 consensus_queue.cc:260] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } }
I20260812 19:08:20.085672 2660 raft_consensus.cc:3037] T d28ed56ce03a49908ad6f468e43b5402 P 6dd313c6592d4f33bab314a73f28b007 [term 5 FOLLOWER]: Advancing to term 6
I20260812 19:08:20.030195 2284 raft_consensus.cc:504] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "dd7e8c7a8bc54e73a39411cfc241fe9c" member_type: VOTER last_known_addr { host: "127.26.205.67" port: 44573 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false } }
I20260812 19:08:20.086216 2861 leader_election.cc:288] T 7adcd8cac12a4350a441ca5a35090ec9 P 182784ca1eb74135b8f16d8894fa4b01 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6dd313c6592d4f33bab314a73f28b007 (127.26.205.65:44295), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:20.073410 29445 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39326 (request call id 2312) took 2131 ms. Trace:
I20260812 19:08:20.087075 29445 rpcz_store.cc:276] 0812 19:08:17.941652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0812 19:08:18.604007 (+662355us) service_pool.cc:227] Handling call
0812 19:08:18.688199 (+ 84192us) raft_consensus.cc:1457] Updating replica for 0 ops
0812 19:08:19.748153 (+1059954us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400283aa0. stack: 00007f9c66ec0033 00007f9c66ebff30 00007f9c6681e534 00007f9c7051e8b4 00007f9c7051e849 00007f9c69d93471 00007f9c69da4951 00007f9c69e05321 00007f9c69e06b4e 00007f9c69e10c89 00007f9c69e11556 00007f9c69e0dedf 00007f9c69e0d99e 00007f9c6ed4f2e9 00007f9c68441684 00007f9c68441612
0812 19:08:20.073071 (+324918us) raft_consensus.cc:1683] Filling consensus response to leader.
0812 19:08:20.073334 (+ 263us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":672115456}
I20260812 19:08:20.088225 2284 leader_election.cc:288] T 47762c70d86d4362baf4ba75331efe8a P dd7e8c7a8bc54e73a39411cfc241fe9c [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), a34c47dd29c14748a018d716ac6c39df (127.26.205.68:41925)
I20260812 19:08:20.053141 2956 consensus_queue.cc:1046] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68" port: 41925 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260812 19:08:20.089362 2874 leader_election.cc:288] T 6e90268109844f62bf1e1ee425b2946a P 6dd313c6592d4f33bab314a73f28b007 [CANDIDATE]: Term 7 election: Requested vote from peers 182784ca1eb74135b8f16d8894fa4b01 (127.26.205.66:43659), dd7e8c7a8bc54e73a39411cfc241fe9c (127.26.205.67:44573)
I20260812 19:08:20.089821 2956 raft_consensus.cc:3032] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Stepping down as leader of term 6
I20260812 19:08:20.070498 29334 raft_consensus.cc:2454] T bf312d0cb2a6414794d20e98e1edf8ba P 6dd313c6592d4f33bab314a73f28b007 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a34c47dd29c14748a018d716ac6c39df in term 8.
I20260812 19:08:20.090229 2956 raft_consensus.cc:729] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 182784ca1eb74135b8f16d8894fa4b01, State: Running, Role: LEADER
I20260812 19:08:20.091168 2956 consensus_queue.cc:260] T 6d8ba27ebf754ac098699f3efab22620 P 182784ca1eb74135b8f16d8894fa4b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "6dd313c6592d4f33bab314a73f28b007" member_type: VOTER last_known_addr { host: "127.26.205.65" port: 44295 } } peers { permanent_uuid: "182784ca1eb74135b8f16d8894fa4b01" member_type: VOTER last_known_addr { host: "127.26.205.66" port: 43659 } } peers { permanent_uuid: "a34c47dd29c14748a018d716ac6c39df" member_type: VOTER last_known_addr { host: "127.26.205.68"